[Bug]: vllm.engine.async_llm_engine.AsyncEngineDeadError: Background loop has errored already.

Your current environment
Collecting environment information...
/data/miniconda3_new/envs/vllm-new/lib/python3.10/site-packages/transformers/utils/hub.py:127: FutureWarning: Using `TRANSFORMERS_CACHE` is deprecated and will be removed in v5 of Transformers. Use `HF_HOME` instead.
  warnings.warn(
PyTorch version: 2.3.0+cu121
Is debug build: False
CUDA used to build PyTorch: 12.1
ROCM used to build PyTorch: N/A

OS: Ubuntu 22.04.4 LTS (x86_64)
GCC version: (Ubuntu 11.4.0-1ubuntu1~22.04) 11.4.0
Clang version: Could not collect
CMake version: version 3.29.3
Libc version: glibc-2.35

Python version: 3.10.0 | packaged by conda-forge | (default, Nov 20 2021, 02:24:10) [GCC 9.4.0] (64-bit runtime)
Python platform: Linux-6.5.0-35-generic-x86_64-with-glibc2.35
Is CUDA available: True
CUDA runtime version: 12.5.40
CUDA_MODULE_LOADING set to: LAZY
GPU models and configuration:
GPU 0: NVIDIA GeForce RTX 3090
GPU 1: NVIDIA GeForce RTX 3090
GPU 2: NVIDIA GeForce RTX 3090
GPU 3: NVIDIA GeForce RTX 3090

Nvidia driver version: 525.147.05
cuDNN version: Could not collect
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

CPU:
架构：                              x86_64
CPU 运行模式：                      32-bit, 64-bit
Address sizes:                      46 bits physical, 48 bits virtual
字节序：                            Little Endian
CPU:                                64
在线 CPU 列表：                     0-63
厂商 ID：                           GenuineIntel
型号名称：                          Intel(R) Xeon(R) Gold 5218 CPU @ 2.30GHz
CPU 系列：                          6
型号：                              85
每个核的线程数：                    2
每个座的核数：                      16
座：                                2
步进：                              7
CPU 最大 MHz：                      3900.0000
CPU 最小 MHz：                      1000.0000
BogoMIPS：                          4600.00
标记：                              fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc art arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf pni pclmulqdq dtes64 ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm 3dnowprefetch cpuid_fault epb cat_l3 cdp_l3 invpcid_single intel_ppin ssbd mba ibrs ibpb stibp ibrs_enhanced tpr_shadow flexpriority ept vpid ept_ad fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid cqm mpx rdt_a avx512f avx512dq rdseed adx smap clflushopt clwb intel_pt avx512cd avx512bw avx512vl xsaveopt xsavec xgetbv1 xsaves cqm_llc cqm_occup_llc cqm_mbm_total cqm_mbm_local dtherm ida arat pln pts vnmi pku ospke avx512_vnni md_clear flush_l1d arch_capabilities
虚拟化：                            VT-x
L1d 缓存：                          1 MiB (32 instances)
L1i 缓存：                          1 MiB (32 instances)
L2 缓存：                           32 MiB (32 instances)
L3 缓存：                           44 MiB (2 instances)
NUMA 节点：                         2
NUMA 节点0 CPU：                    0-15,32-47
NUMA 节点1 CPU：                    16-31,48-63
Vulnerability Gather data sampling: Mitigation; Microcode
Vulnerability Itlb multihit:        KVM: Mitigation: VMX disabled
Vulnerability L1tf:                 Not affected
Vulnerability Mds:                  Not affected
Vulnerability Meltdown:             Not affected
Vulnerability Mmio stale data:      Mitigation; Clear CPU buffers; SMT vulnerable
Vulnerability Retbleed:             Mitigation; Enhanced IBRS
Vulnerability Spec rstack overflow: Not affected
Vulnerability Spec store bypass:    Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1:           Mitigation; usercopy/swapgs barriers and __user pointer sanitization
Vulnerability Spectre v2:           Mitigation; Enhanced / Automatic IBRS; IBPB conditional; RSB filling; PBRSB-eIBRS SW sequence; BHI Syscall hardening, KVM SW loop
Vulnerability Srbds:                Not affected
Vulnerability Tsx async abort:      Mitigation; TSX disabled

Versions of relevant libraries:
[pip3] flashinfer==0.0.8+cu121torch2.3
[pip3] numpy==1.26.4
[pip3] torch==2.3.0
[pip3] torchvision==0.18.0
[pip3] triton==2.3.0
[conda] flashinfer                0.0.8+cu121torch2.3          pypi_0    pypi
[conda] numpy                     1.26.4                   pypi_0    pypi
[conda] torch                     2.3.0                    pypi_0    pypi
[conda] torchvision               0.18.0                   pypi_0    pypi
[conda] triton                    2.3.0                    pypi_0    pypiROCM Version: Could not collect
Neuron SDK Version: N/A
vLLM Version: 0.5.1
vLLM Build Flags:
CUDA Archs: Not Set; ROCm: Disabled; Neuron: Disabled
GPU Topology:
GPU0	GPU1	GPU2	GPU3	NIC0	NIC1	CPU Affinity	NUMA Affinity
GPU0	 X 	PIX	NODE	NODE	NODE	NODE	0-15,32-47	0
GPU1	PIX	 X 	NODE	NODE	NODE	NODE	0-15,32-47	0
GPU2	NODE	NODE	 X 	PIX	NODE	NODE	0-15,32-47	0
GPU3	NODE	NODE	PIX	 X 	NODE	NODE	0-15,32-47	0
NIC0	NODE	NODE	NODE	NODE	 X 	PIX
NIC1	NODE	NODE	NODE	NODE	PIX	 X

Legend:

  X    = Self
  SYS  = Connection traversing PCIe as well as the SMP interconnect between NUMA nodes (e.g., QPI/UPI)
  NODE = Connection traversing PCIe as well as the interconnect between PCIe Host Bridges within a NUMA node
  PHB  = Connection traversing PCIe as well as a PCIe Host Bridge (typically the CPU)
  PXB  = Connection traversing multiple PCIe bridges (without traversing the PCIe Host Bridge)
  PIX  = Connection traversing at most a single PCIe bridge
  NV#  = Connection traversing a bonded set of # NVLinks

NIC Legend:

  NIC0: mlx5_0
  NIC1: mlx5_1

🐛 Describe the bug
32 concurrent massive inferences
  File "/data/miniconda3_new/envs/vllm-new/lib/python3.10/site-packages/starlette/routing.py", line 72, in app
    response = await func(request)
  File "/data/miniconda3_new/envs/vllm-new/lib/python3.10/site-packages/fastapi/routing.py", line 278, in app
    raw_response = await run_endpoint_function(
  File "/data/miniconda3_new/envs/vllm-new/lib/python3.10/site-packages/fastapi/routing.py", line 191, in run_endpoint_function
    return await dependant.call(**values)
  File "/data/miniconda3_new/envs/vllm-new/lib/python3.10/site-packages/vllm/entrypoints/openai/api_server.py", line 132, in create_chat_completion
    generator = await openai_serving_chat.create_chat_completion(
  File "/data/miniconda3_new/envs/vllm-new/lib/python3.10/site-packages/vllm/entrypoints/openai/serving_chat.py", line 305, in create_chat_completion
    return await self.chat_completion_full_generator(
  File "/data/miniconda3_new/envs/vllm-new/lib/python3.10/site-packages/vllm/entrypoints/openai/serving_chat.py", line 505, in chat_completion_full_generator
    async for res in result_generator:
  File "/data/miniconda3_new/envs/vllm-new/lib/python3.10/site-packages/vllm/engine/async_llm_engine.py", line 746, in generate
    async for output in self._process_request(
  File "/data/miniconda3_new/envs/vllm-new/lib/python3.10/site-packages/vllm/engine/async_llm_engine.py", line 859, in _process_request
    raise e
  File "/data/miniconda3_new/envs/vllm-new/lib/python3.10/site-packages/vllm/engine/async_llm_engine.py", line 855, in _process_request
    async for request_output in stream:
  File "/data/miniconda3_new/envs/vllm-new/lib/python3.10/site-packages/vllm/engine/async_llm_engine.py", line 90, in __anext__
    raise result
  File "/data/miniconda3_new/envs/vllm-new/lib/python3.10/site-packages/uvicorn/protocols/http/httptools_impl.py", line 411, in run_asgi
    result = await app(  # type: ignore[func-returns-value]
  File "/data/miniconda3_new/envs/vllm-new/lib/python3.10/site-packages/uvicorn/middleware/proxy_headers.py", line 69, in __call__
    return await self.app(scope, receive, send)
  File "/data/miniconda3_new/envs/vllm-new/lib/python3.10/site-packages/fastapi/applications.py", line 1054, in __call__
    await super().__call__(scope, receive, send)
  File "/data/miniconda3_new/envs/vllm-new/lib/python3.10/site-packages/starlette/applications.py", line 123, in __call__
    await self.middleware_stack(scope, receive, send)
  File "/data/miniconda3_new/envs/vllm-new/lib/python3.10/site-packages/starlette/middleware/errors.py", line 186, in __call__
    raise exc
  File "/data/miniconda3_new/envs/vllm-new/lib/python3.10/site-packages/starlette/middleware/errors.py", line 164, in __call__
    await self.app(scope, receive, _send)
  File "/data/miniconda3_new/envs/vllm-new/lib/python3.10/site-packages/starlette/middleware/cors.py", line 85, in __call__
    await self.app(scope, receive, send)
  File "/data/miniconda3_new/envs/vllm-new/lib/python3.10/site-packages/starlette/middleware/exceptions.py", line 65, in __call__
    await wrap_app_handling_exceptions(self.app, conn)(scope, receive, send)
  File "/data/miniconda3_new/envs/vllm-new/lib/python3.10/site-packages/starlette/_exception_handler.py", line 64, in wrapped_app
    raise exc
  File "/data/miniconda3_new/envs/vllm-new/lib/python3.10/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
    await app(scope, receive, sender)
  File "/data/miniconda3_new/envs/vllm-new/lib/python3.10/site-packages/starlette/routing.py", line 756, in __call__
    await self.middleware_stack(scope, receive, send)
  File "/data/miniconda3_new/envs/vllm-new/lib/python3.10/site-packages/starlette/routing.py", line 776, in app
    await route.handle(scope, receive, send)
  File "/data/miniconda3_new/envs/vllm-new/lib/python3.10/site-packages/starlette/routing.py", line 297, in handle
    await self.app(scope, receive, send)
  File "/data/miniconda3_new/envs/vllm-new/lib/python3.10/site-packages/starlette/routing.py", line 77, in app
    await wrap_app_handling_exceptions(app, request)(scope, receive, send)
  File "/data/miniconda3_new/envs/vllm-new/lib/python3.10/site-packages/starlette/_exception_handler.py", line 64, in wrapped_app
    raise exc
  File "/data/miniconda3_new/envs/vllm-new/lib/python3.10/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
    await app(scope, receive, sender)
  File "/data/miniconda3_new/envs/vllm-new/lib/python3.10/site-packages/starlette/routing.py", line 72, in app
    response = await func(request)
  File "/data/miniconda3_new/envs/vllm-new/lib/python3.10/site-packages/fastapi/routing.py", line 278, in app
    raw_response = await run_endpoint_function(
  File "/data/miniconda3_new/envs/vllm-new/lib/python3.10/site-packages/fastapi/routing.py", line 191, in run_endpoint_function
    return await dependant.call(**values)
  File "/data/miniconda3_new/envs/vllm-new/lib/python3.10/site-packages/vllm/entrypoints/openai/api_server.py", line 132, in create_chat_completion
    generator = await openai_serving_chat.create_chat_completion(
  File "/data/miniconda3_new/envs/vllm-new/lib/python3.10/site-packages/vllm/entrypoints/openai/serving_chat.py", line 305, in create_chat_completion
    return await self.chat_completion_full_generator(
  File "/data/miniconda3_new/envs/vllm-new/lib/python3.10/site-packages/vllm/entrypoints/openai/serving_chat.py", line 505, in chat_completion_full_generator
    async for res in result_generator:
  File "/data/miniconda3_new/envs/vllm-new/lib/python3.10/site-packages/vllm/engine/async_llm_engine.py", line 746, in generate
    async for output in self._process_request(
  File "/data/miniconda3_new/envs/vllm-new/lib/python3.10/site-packages/vllm/engine/async_llm_engine.py", line 859, in _process_request
    raise e
  File "/data/miniconda3_new/envs/vllm-new/lib/python3.10/site-packages/vllm/engine/async_llm_engine.py", line 855, in _process_request
    async for request_output in stream:
  File "/data/miniconda3_new/envs/vllm-new/lib/python3.10/site-packages/vllm/engine/async_llm_engine.py", line 90, in __anext__
    raise result
  File "/data/miniconda3_new/envs/vllm-new/lib/python3.10/site-packages/uvicorn/protocols/http/httptools_impl.py", line 411, in run_asgi
    result = await app(  # type: ignore[func-returns-value]
  File "/data/miniconda3_new/envs/vllm-new/lib/python3.10/site-packages/uvicorn/middleware/proxy_headers.py", line 69, in __call__
    return await self.app(scope, receive, send)
  File "/data/miniconda3_new/envs/vllm-new/lib/python3.10/site-packages/fastapi/applications.py", line 1054, in __call__
    await super().__call__(scope, receive, send)
  File "/data/miniconda3_new/envs/vllm-new/lib/python3.10/site-packages/starlette/applications.py", line 123, in __call__
    await self.middleware_stack(scope, receive, send)
  File "/data/miniconda3_new/envs/vllm-new/lib/python3.10/site-packages/starlette/middleware/errors.py", line 186, in __call__
    raise exc
  File "/data/miniconda3_new/envs/vllm-new/lib/python3.10/site-packages/starlette/middleware/errors.py", line 164, in __call__
    await self.app(scope, receive, _send)
  File "/data/miniconda3_new/envs/vllm-new/lib/python3.10/site-packages/starlette/middleware/cors.py", line 85, in __call__
    await self.app(scope, receive, send)
  File "/data/miniconda3_new/envs/vllm-new/lib/python3.10/site-packages/starlette/middleware/exceptions.py", line 65, in __call__
    await wrap_app_handling_exceptions(self.app, conn)(scope, receive, send)
  File "/data/miniconda3_new/envs/vllm-new/lib/python3.10/site-packages/starlette/_exception_handler.py", line 64, in wrapped_app
    raise exc
  File "/data/miniconda3_new/envs/vllm-new/lib/python3.10/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
    await app(scope, receive, sender)
  File "/data/miniconda3_new/envs/vllm-new/lib/python3.10/site-packages/starlette/routing.py", line 756, in __call__
    await self.middleware_stack(scope, receive, send)
  File "/data/miniconda3_new/envs/vllm-new/lib/python3.10/site-packages/starlette/routing.py", line 776, in app
    await route.handle(scope, receive, send)
  File "/data/miniconda3_new/envs/vllm-new/lib/python3.10/site-packages/starlette/routing.py", line 297, in handle
    await self.app(scope, receive, send)
  File "/data/miniconda3_new/envs/vllm-new/lib/python3.10/site-packages/starlette/routing.py", line 77, in app
    await wrap_app_handling_exceptions(app, request)(scope, receive, send)
  File "/data/miniconda3_new/envs/vllm-new/lib/python3.10/site-packages/starlette/_exception_handler.py", line 64, in wrapped_app
    raise exc
  File "/data/miniconda3_new/envs/vllm-new/lib/python3.10/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
    await app(scope, receive, sender)
  File "/data/miniconda3_new/envs/vllm-new/lib/python3.10/site-packages/starlette/routing.py", line 72, in app
    response = await func(request)
  File "/data/miniconda3_new/envs/vllm-new/lib/python3.10/site-packages/fastapi/routing.py", line 278, in app
    raw_response = await run_endpoint_function(
  File "/data/miniconda3_new/envs/vllm-new/lib/python3.10/site-packages/fastapi/routing.py", line 191, in run_endpoint_function
    return await dependant.call(**values)
  File "/data/miniconda3_new/envs/vllm-new/lib/python3.10/site-packages/vllm/entrypoints/openai/api_server.py", line 132, in create_chat_completion
    generator = await openai_serving_chat.create_chat_completion(
  File "/data/miniconda3_new/envs/vllm-new/lib/python3.10/site-packages/vllm/entrypoints/openai/serving_chat.py", line 305, in create_chat_completion
    return await self.chat_completion_full_generator(
  File "/data/miniconda3_new/envs/vllm-new/lib/python3.10/site-packages/vllm/entrypoints/openai/serving_chat.py", line 505, in chat_completion_full_generator
    async for res in result_generator:
  File "/data/miniconda3_new/envs/vllm-new/lib/python3.10/site-packages/vllm/engine/async_llm_engine.py", line 746, in generate
    async for output in self._process_request(
  File "/data/miniconda3_new/envs/vllm-new/lib/python3.10/site-packages/vllm/engine/async_llm_engine.py", line 859, in _process_request
    raise e
  File "/data/miniconda3_new/envs/vllm-new/lib/python3.10/site-packages/vllm/engine/async_llm_engine.py", line 855, in _process_request
    async for request_output in stream:
  File "/data/miniconda3_new/envs/vllm-new/lib/python3.10/site-packages/vllm/engine/async_llm_engine.py", line 90, in __anext__
    raise result
  File "/data/miniconda3_new/envs/vllm-new/lib/python3.10/site-packages/uvicorn/protocols/http/httptools_impl.py", line 411, in run_asgi
    result = await app(  # type: ignore[func-returns-value]
  File "/data/miniconda3_new/envs/vllm-new/lib/python3.10/site-packages/uvicorn/middleware/proxy_headers.py", line 69, in __call__
    return await self.app(scope, receive, send)
  File "/data/miniconda3_new/envs/vllm-new/lib/python3.10/site-packages/fastapi/applications.py", line 1054, in __call__
    await super().__call__(scope, receive, send)
  File "/data/miniconda3_new/envs/vllm-new/lib/python3.10/site-packages/starlette/applications.py", line 123, in __call__
    await self.middleware_stack(scope, receive, send)
  File "/data/miniconda3_new/envs/vllm-new/lib/python3.10/site-packages/starlette/middleware/errors.py", line 186, in __call__
    raise exc
  File "/data/miniconda3_new/envs/vllm-new/lib/python3.10/site-packages/starlette/middleware/errors.py", line 164, in __call__
    await self.app(scope, receive, _send)
  File "/data/miniconda3_new/envs/vllm-new/lib/python3.10/site-packages/starlette/middleware/cors.py", line 85, in __call__
    await self.app(scope, receive, send)
  File "/data/miniconda3_new/envs/vllm-new/lib/python3.10/site-packages/starlette/middleware/exceptions.py", line 65, in __call__
    await wrap_app_handling_exceptions(self.app, conn)(scope, receive, send)
  File "/data/miniconda3_new/envs/vllm-new/lib/python3.10/site-packages/starlette/_exception_handler.py", line 64, in wrapped_app
    raise exc
  File "/data/miniconda3_new/envs/vllm-new/lib/python3.10/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
    await app(scope, receive, sender)
  File "/data/miniconda3_new/envs/vllm-new/lib/python3.10/site-packages/starlette/routing.py", line 756, in __call__
    await self.middleware_stack(scope, receive, send)
  File "/data/miniconda3_new/envs/vllm-new/lib/python3.10/site-packages/starlette/routing.py", line 776, in app
    await route.handle(scope, receive, send)
  File "/data/miniconda3_new/envs/vllm-new/lib/python3.10/site-packages/starlette/routing.py", line 297, in handle
    await self.app(scope, receive, send)
  File "/data/miniconda3_new/envs/vllm-new/lib/python3.10/site-packages/starlette/routing.py", line 77, in app
    await wrap_app_handling_exceptions(app, request)(scope, receive, send)
  File "/data/miniconda3_new/envs/vllm-new/lib/python3.10/site-packages/starlette/_exception_handler.py", line 64, in wrapped_app
    raise exc
  File "/data/miniconda3_new/envs/vllm-new/lib/python3.10/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
    await app(scope, receive, sender)
  File "/data/miniconda3_new/envs/vllm-new/lib/python3.10/site-packages/starlette/routing.py", line 72, in app
    response = await func(request)
  File "/data/miniconda3_new/envs/vllm-new/lib/python3.10/site-packages/fastapi/routing.py", line 278, in app
    raw_response = await run_endpoint_function(
  File "/data/miniconda3_new/envs/vllm-new/lib/python3.10/site-packages/fastapi/routing.py", line 191, in run_endpoint_function
    return await dependant.call(**values)
  File "/data/miniconda3_new/envs/vllm-new/lib/python3.10/site-packages/vllm/entrypoints/openai/api_server.py", line 132, in create_chat_completion
    generator = await openai_serving_chat.create_chat_completion(
  File "/data/miniconda3_new/envs/vllm-new/lib/python3.10/site-packages/vllm/entrypoints/openai/serving_chat.py", line 305, in create_chat_completion
    return await self.chat_completion_full_generator(
  File "/data/miniconda3_new/envs/vllm-new/lib/python3.10/site-packages/vllm/entrypoints/openai/serving_chat.py", line 505, in chat_completion_full_generator
    async for res in result_generator:
  File "/data/miniconda3_new/envs/vllm-new/lib/python3.10/site-packages/vllm/engine/async_llm_engine.py", line 746, in generate
    async for output in self._process_request(
  File "/data/miniconda3_new/envs/vllm-new/lib/python3.10/site-packages/vllm/engine/async_llm_engine.py", line 859, in _process_request
    raise e
  File "/data/miniconda3_new/envs/vllm-new/lib/python3.10/site-packages/vllm/engine/async_llm_engine.py", line 855, in _process_request
    async for request_output in stream:
  File "/data/miniconda3_new/envs/vllm-new/lib/python3.10/site-packages/vllm/engine/async_llm_engine.py", line 90, in __anext__
    raise result
  File "/data/miniconda3_new/envs/vllm-new/lib/python3.10/site-packages/uvicorn/protocols/http/httptools_impl.py", line 411, in run_asgi
    result = await app(  # type: ignore[func-returns-value]
  File "/data/miniconda3_new/envs/vllm-new/lib/python3.10/site-packages/uvicorn/middleware/proxy_headers.py", line 69, in __call__
    return await self.app(scope, receive, send)
  File "/data/miniconda3_new/envs/vllm-new/lib/python3.10/site-packages/fastapi/applications.py", line 1054, in __call__
    await super().__call__(scope, receive, send)
  File "/data/miniconda3_new/envs/vllm-new/lib/python3.10/site-packages/starlette/applications.py", line 123, in __call__
    await self.middleware_stack(scope, receive, send)
  File "/data/miniconda3_new/envs/vllm-new/lib/python3.10/site-packages/starlette/middleware/errors.py", line 186, in __call__
    raise exc
  File "/data/miniconda3_new/envs/vllm-new/lib/python3.10/site-packages/starlette/middleware/errors.py", line 164, in __call__
    await self.app(scope, receive, _send)
  File "/data/miniconda3_new/envs/vllm-new/lib/python3.10/site-packages/starlette/middleware/cors.py", line 85, in __call__
    await self.app(scope, receive, send)
  File "/data/miniconda3_new/envs/vllm-new/lib/python3.10/site-packages/starlette/middleware/exceptions.py", line 65, in __call__
    await wrap_app_handling_exceptions(self.app, conn)(scope, receive, send)
  File "/data/miniconda3_new/envs/vllm-new/lib/python3.10/site-packages/starlette/_exception_handler.py", line 64, in wrapped_app
    raise exc
  File "/data/miniconda3_new/envs/vllm-new/lib/python3.10/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
    await app(scope, receive, sender)
  File "/data/miniconda3_new/envs/vllm-new/lib/python3.10/site-packages/starlette/routing.py", line 756, in __call__
    await self.middleware_stack(scope, receive, send)
  File "/data/miniconda3_new/envs/vllm-new/lib/python3.10/site-packages/starlette/routing.py", line 776, in app
    await route.handle(scope, receive, send)
  File "/data/miniconda3_new/envs/vllm-new/lib/python3.10/site-packages/starlette/routing.py", line 297, in handle
    await self.app(scope, receive, send)
  File "/data/miniconda3_new/envs/vllm-new/lib/python3.10/site-packages/starlette/routing.py", line 77, in app
    await wrap_app_handling_exceptions(app, request)(scope, receive, send)
  File "/data/miniconda3_new/envs/vllm-new/lib/python3.10/site-packages/starlette/_exception_handler.py", line 64, in wrapped_app
    raise exc
  File "/data/miniconda3_new/envs/vllm-new/lib/python3.10/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
    await app(scope, receive, sender)
  File "/data/miniconda3_new/envs/vllm-new/lib/python3.10/site-packages/starlette/routing.py", line 72, in app
    response = await func(request)
  File "/data/miniconda3_new/envs/vllm-new/lib/python3.10/site-packages/fastapi/routing.py", line 278, in app
    raw_response = await run_endpoint_function(
  File "/data/miniconda3_new/envs/vllm-new/lib/python3.10/site-packages/fastapi/routing.py", line 191, in run_endpoint_function
    return await dependant.call(**values)
  File "/data/miniconda3_new/envs/vllm-new/lib/python3.10/site-packages/vllm/entrypoints/openai/api_server.py", line 132, in create_chat_completion
    generator = await openai_serving_chat.create_chat_completion(
  File "/data/miniconda3_new/envs/vllm-new/lib/python3.10/site-packages/vllm/entrypoints/openai/serving_chat.py", line 305, in create_chat_completion
    return await self.chat_completion_full_generator(
  File "/data/miniconda3_new/envs/vllm-new/lib/python3.10/site-packages/vllm/entrypoints/openai/serving_chat.py", line 505, in chat_completion_full_generator
    async for res in result_generator:
  File "/data/miniconda3_new/envs/vllm-new/lib/python3.10/site-packages/vllm/engine/async_llm_engine.py", line 746, in generate
    async for output in self._process_request(
  File "/data/miniconda3_new/envs/vllm-new/lib/python3.10/site-packages/vllm/engine/async_llm_engine.py", line 859, in _process_request
    raise e
  File "/data/miniconda3_new/envs/vllm-new/lib/python3.10/site-packages/vllm/engine/async_llm_engine.py", line 855, in _process_request
    async for request_output in stream:
  File "/data/miniconda3_new/envs/vllm-new/lib/python3.10/site-packages/vllm/engine/async_llm_engine.py", line 90, in __anext__
    raise result
  File "/data/miniconda3_new/envs/vllm-new/lib/python3.10/site-packages/uvicorn/protocols/http/httptools_impl.py", line 411, in run_asgi
    result = await app(  # type: ignore[func-returns-value]
  File "/data/miniconda3_new/envs/vllm-new/lib/python3.10/site-packages/uvicorn/middleware/proxy_headers.py", line 69, in __call__
    return await self.app(scope, receive, send)
  File "/data/miniconda3_new/envs/vllm-new/lib/python3.10/site-packages/fastapi/applications.py", line 1054, in __call__
    await super().__call__(scope, receive, send)
  File "/data/miniconda3_new/envs/vllm-new/lib/python3.10/site-packages/starlette/applications.py", line 123, in __call__
    await self.middleware_stack(scope, receive, send)
  File "/data/miniconda3_new/envs/vllm-new/lib/python3.10/site-packages/starlette/middleware/errors.py", line 186, in __call__
    raise exc
  File "/data/miniconda3_new/envs/vllm-new/lib/python3.10/site-packages/starlette/middleware/errors.py", line 164, in __call__
    await self.app(scope, receive, _send)
  File "/data/miniconda3_new/envs/vllm-new/lib/python3.10/site-packages/starlette/middleware/cors.py", line 85, in __call__
    await self.app(scope, receive, send)
  File "/data/miniconda3_new/envs/vllm-new/lib/python3.10/site-packages/starlette/middleware/exceptions.py", line 65, in __call__
    await wrap_app_handling_exceptions(self.app, conn)(scope, receive, send)
  File "/data/miniconda3_new/envs/vllm-new/lib/python3.10/site-packages/starlette/_exception_handler.py", line 64, in wrapped_app
    raise exc
  File "/data/miniconda3_new/envs/vllm-new/lib/python3.10/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
    await app(scope, receive, sender)
  File "/data/miniconda3_new/envs/vllm-new/lib/python3.10/site-packages/starlette/routing.py", line 756, in __call__
    await self.middleware_stack(scope, receive, send)
  File "/data/miniconda3_new/envs/vllm-new/lib/python3.10/site-packages/starlette/routing.py", line 776, in app
    await route.handle(scope, receive, send)
  File "/data/miniconda3_new/envs/vllm-new/lib/python3.10/site-packages/starlette/routing.py", line 297, in handle
    await self.app(scope, receive, send)
  File "/data/miniconda3_new/envs/vllm-new/lib/python3.10/site-packages/starlette/routing.py", line 77, in app
    await wrap_app_handling_exceptions(app, request)(scope, receive, send)
  File "/data/miniconda3_new/envs/vllm-new/lib/python3.10/site-packages/starlette/_exception_handler.py", line 64, in wrapped_app
    raise exc
  File "/data/miniconda3_new/envs/vllm-new/lib/python3.10/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
    await app(scope, receive, sender)
  File "/data/miniconda3_new/envs/vllm-new/lib/python3.10/site-packages/starlette/routing.py", line 72, in app
    response = await func(request)
  File "/data/miniconda3_new/envs/vllm-new/lib/python3.10/site-packages/fastapi/routing.py", line 278, in app
    raw_response = await run_endpoint_function(
  File "/data/miniconda3_new/envs/vllm-new/lib/python3.10/site-packages/fastapi/routing.py", line 191, in run_endpoint_function
    return await dependant.call(**values)
  File "/data/miniconda3_new/envs/vllm-new/lib/python3.10/site-packages/vllm/entrypoints/openai/api_server.py", line 132, in create_chat_completion
    generator = await openai_serving_chat.create_chat_completion(
  File "/data/miniconda3_new/envs/vllm-new/lib/python3.10/site-packages/vllm/entrypoints/openai/serving_chat.py", line 305, in create_chat_completion
    return await self.chat_completion_full_generator(
  File "/data/miniconda3_new/envs/vllm-new/lib/python3.10/site-packages/vllm/entrypoints/openai/serving_chat.py", line 505, in chat_completion_full_generator
    async for res in result_generator:
  File "/data/miniconda3_new/envs/vllm-new/lib/python3.10/site-packages/vllm/engine/async_llm_engine.py", line 746, in generate
    async for output in self._process_request(
  File "/data/miniconda3_new/envs/vllm-new/lib/python3.10/site-packages/vllm/engine/async_llm_engine.py", line 859, in _process_request
    raise e
  File "/data/miniconda3_new/envs/vllm-new/lib/python3.10/site-packages/vllm/engine/async_llm_engine.py", line 855, in _process_request
    async for request_output in stream:
  File "/data/miniconda3_new/envs/vllm-new/lib/python3.10/site-packages/vllm/engine/async_llm_engine.py", line 90, in __anext__
    raise result
  File "/data/miniconda3_new/envs/vllm-new/lib/python3.10/site-packages/uvicorn/protocols/http/httptools_impl.py", line 411, in run_asgi
    result = await app(  # type: ignore[func-returns-value]
  File "/data/miniconda3_new/envs/vllm-new/lib/python3.10/site-packages/uvicorn/middleware/proxy_headers.py", line 69, in __call__
    return await self.app(scope, receive, send)
  File "/data/miniconda3_new/envs/vllm-new/lib/python3.10/site-packages/fastapi/applications.py", line 1054, in __call__
    await super().__call__(scope, receive, send)
  File "/data/miniconda3_new/envs/vllm-new/lib/python3.10/site-packages/starlette/applications.py", line 123, in __call__
    await self.middleware_stack(scope, receive, send)
  File "/data/miniconda3_new/envs/vllm-new/lib/python3.10/site-packages/starlette/middleware/errors.py", line 186, in __call__
    raise exc
  File "/data/miniconda3_new/envs/vllm-new/lib/python3.10/site-packages/starlette/middleware/errors.py", line 164, in __call__
    await self.app(scope, receive, _send)
  File "/data/miniconda3_new/envs/vllm-new/lib/python3.10/site-packages/starlette/middleware/cors.py", line 85, in __call__
    await self.app(scope, receive, send)
  File "/data/miniconda3_new/envs/vllm-new/lib/python3.10/site-packages/starlette/middleware/exceptions.py", line 65, in __call__
    await wrap_app_handling_exceptions(self.app, conn)(scope, receive, send)
  File "/data/miniconda3_new/envs/vllm-new/lib/python3.10/site-packages/starlette/_exception_handler.py", line 64, in wrapped_app
    raise exc
  File "/data/miniconda3_new/envs/vllm-new/lib/python3.10/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
    await app(scope, receive, sender)
  File "/data/miniconda3_new/envs/vllm-new/lib/python3.10/site-packages/starlette/routing.py", line 756, in __call__
    await self.middleware_stack(scope, receive, send)
  File "/data/miniconda3_new/envs/vllm-new/lib/python3.10/site-packages/starlette/routing.py", line 776, in app
    await route.handle(scope, receive, send)
  File "/data/miniconda3_new/envs/vllm-new/lib/python3.10/site-packages/starlette/routing.py", line 297, in handle
    await self.app(scope, receive, send)
  File "/data/miniconda3_new/envs/vllm-new/lib/python3.10/site-packages/starlette/routing.py", line 77, in app
    await wrap_app_handling_exceptions(app, request)(scope, receive, send)
  File "/data/miniconda3_new/envs/vllm-new/lib/python3.10/site-packages/starlette/_exception_handler.py", line 64, in wrapped_app
    raise exc
  File "/data/miniconda3_new/envs/vllm-new/lib/python3.10/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
    await app(scope, receive, sender)
  File "/data/miniconda3_new/envs/vllm-new/lib/python3.10/site-packages/starlette/routing.py", line 72, in app
    response = await func(request)
  File "/data/miniconda3_new/envs/vllm-new/lib/python3.10/site-packages/fastapi/routing.py", line 278, in app
    raw_response = await run_endpoint_function(
  File "/data/miniconda3_new/envs/vllm-new/lib/python3.10/site-packages/fastapi/routing.py", line 191, in run_endpoint_function
    return await dependant.call(**values)
  File "/data/miniconda3_new/envs/vllm-new/lib/python3.10/site-packages/vllm/entrypoints/openai/api_server.py", line 132, in create_chat_completion
    generator = await openai_serving_chat.create_chat_completion(
  File "/data/miniconda3_new/envs/vllm-new/lib/python3.10/site-packages/vllm/entrypoints/openai/serving_chat.py", line 305, in create_chat_completion
    return await self.chat_completion_full_generator(
  File "/data/miniconda3_new/envs/vllm-new/lib/python3.10/site-packages/vllm/entrypoints/openai/serving_chat.py", line 505, in chat_completion_full_generator
    async for res in result_generator:
  File "/data/miniconda3_new/envs/vllm-new/lib/python3.10/site-packages/vllm/engine/async_llm_engine.py", line 746, in generate
    async for output in self._process_request(
  File "/data/miniconda3_new/envs/vllm-new/lib/python3.10/site-packages/vllm/engine/async_llm_engine.py", line 859, in _process_request
    raise e
  File "/data/miniconda3_new/envs/vllm-new/lib/python3.10/site-packages/vllm/engine/async_llm_engine.py", line 855, in _process_request
    async for request_output in stream:
  File "/data/miniconda3_new/envs/vllm-new/lib/python3.10/site-packages/vllm/engine/async_llm_engine.py", line 90, in __anext__
    raise result
  File "/data/miniconda3_new/envs/vllm-new/lib/python3.10/site-packages/uvicorn/protocols/http/httptools_impl.py", line 411, in run_asgi
    result = await app(  # type: ignore[func-returns-value]
  File "/data/miniconda3_new/envs/vllm-new/lib/python3.10/site-packages/uvicorn/middleware/proxy_headers.py", line 69, in __call__
    return await self.app(scope, receive, send)
  File "/data/miniconda3_new/envs/vllm-new/lib/python3.10/site-packages/fastapi/applications.py", line 1054, in __call__
    await super().__call__(scope, receive, send)
  File "/data/miniconda3_new/envs/vllm-new/lib/python3.10/site-packages/starlette/applications.py", line 123, in __call__
    await self.middleware_stack(scope, receive, send)
  File "/data/miniconda3_new/envs/vllm-new/lib/python3.10/site-packages/starlette/middleware/errors.py", line 186, in __call__
    raise exc
  File "/data/miniconda3_new/envs/vllm-new/lib/python3.10/site-packages/starlette/middleware/errors.py", line 164, in __call__
    await self.app(scope, receive, _send)
  File "/data/miniconda3_new/envs/vllm-new/lib/python3.10/site-packages/starlette/middleware/cors.py", line 85, in __call__
    await self.app(scope, receive, send)
  File "/data/miniconda3_new/envs/vllm-new/lib/python3.10/site-packages/starlette/middleware/exceptions.py", line 65, in __call__
    await wrap_app_handling_exceptions(self.app, conn)(scope, receive, send)
  File "/data/miniconda3_new/envs/vllm-new/lib/python3.10/site-packages/starlette/_exception_handler.py", line 64, in wrapped_app
    raise exc
  File "/data/miniconda3_new/envs/vllm-new/lib/python3.10/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
    await app(scope, receive, sender)
  File "/data/miniconda3_new/envs/vllm-new/lib/python3.10/site-packages/starlette/routing.py", line 756, in __call__
    await self.middleware_stack(scope, receive, send)
  File "/data/miniconda3_new/envs/vllm-new/lib/python3.10/site-packages/starlette/routing.py", line 776, in app
    await route.handle(scope, receive, send)
  File "/data/miniconda3_new/envs/vllm-new/lib/python3.10/site-packages/starlette/routing.py", line 297, in handle
    await self.app(scope, receive, send)
  File "/data/miniconda3_new/envs/vllm-new/lib/python3.10/site-packages/starlette/routing.py", line 77, in app
    await wrap_app_handling_exceptions(app, request)(scope, receive, send)
  File "/data/miniconda3_new/envs/vllm-new/lib/python3.10/site-packages/starlette/_exception_handler.py", line 64, in wrapped_app
    raise exc
  File "/data/miniconda3_new/envs/vllm-new/lib/python3.10/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
    await app(scope, receive, sender)
  File "/data/miniconda3_new/envs/vllm-new/lib/python3.10/site-packages/starlette/routing.py", line 72, in app
    response = await func(request)
  File "/data/miniconda3_new/envs/vllm-new/lib/python3.10/site-packages/fastapi/routing.py", line 278, in app
    raw_response = await run_endpoint_function(
  File "/data/miniconda3_new/envs/vllm-new/lib/python3.10/site-packages/fastapi/routing.py", line 191, in run_endpoint_function
    return await dependant.call(**values)
  File "/data/miniconda3_new/envs/vllm-new/lib/python3.10/site-packages/vllm/entrypoints/openai/api_server.py", line 132, in create_chat_completion
    generator = await openai_serving_chat.create_chat_completion(
  File "/data/miniconda3_new/envs/vllm-new/lib/python3.10/site-packages/vllm/entrypoints/openai/serving_chat.py", line 305, in create_chat_completion
    return await self.chat_completion_full_generator(
  File "/data/miniconda3_new/envs/vllm-new/lib/python3.10/site-packages/vllm/entrypoints/openai/serving_chat.py", line 505, in chat_completion_full_generator
    async for res in result_generator:
  File "/data/miniconda3_new/envs/vllm-new/lib/python3.10/site-packages/vllm/engine/async_llm_engine.py", line 746, in generate
    async for output in self._process_request(
  File "/data/miniconda3_new/envs/vllm-new/lib/python3.10/site-packages/vllm/engine/async_llm_engine.py", line 859, in _process_request
    raise e
  File "/data/miniconda3_new/envs/vllm-new/lib/python3.10/site-packages/vllm/engine/async_llm_engine.py", line 855, in _process_request
    async for request_output in stream:
  File "/data/miniconda3_new/envs/vllm-new/lib/python3.10/site-packages/vllm/engine/async_llm_engine.py", line 90, in __anext__
    raise result
  File "/data/miniconda3_new/envs/vllm-new/lib/python3.10/site-packages/vllm/engine/async_llm_engine.py", line 43, in _log_task_completion
    return_value = task.result()
  File "/data/miniconda3_new/envs/vllm-new/lib/python3.10/site-packages/vllm/engine/async_llm_engine.py", line 595, in run_engine_loop
    result = task.result()
  File "/data/miniconda3_new/envs/vllm-new/lib/python3.10/site-packages/vllm/engine/async_llm_engine.py", line 540, in engine_step
    request_outputs = await self.engine.step_async(virtual_engine)
  File "/data/miniconda3_new/envs/vllm-new/lib/python3.10/site-packages/vllm/engine/async_llm_engine.py", line 241, in step_async
    output = await self.model_executor.execute_model_async(
  File "/data/miniconda3_new/envs/vllm-new/lib/python3.10/site-packages/vllm/executor/distributed_gpu_executor.py", line 173, in execute_model_async
    return await self._driver_execute_model_async(execute_model_req)
  File "/data/miniconda3_new/envs/vllm-new/lib/python3.10/site-packages/vllm/executor/multiproc_gpu_executor.py", line 160, in _driver_execute_model_async
    return await self.driver_exec_model(execute_model_req)
  File "/data/miniconda3_new/envs/vllm-new/lib/python3.10/concurrent/futures/thread.py", line 52, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/data/miniconda3_new/envs/vllm-new/lib/python3.10/site-packages/vllm/worker/worker_base.py", line 260, in execute_model
    self.execute_worker(worker_input)
  File "/data/miniconda3_new/envs/vllm-new/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
  File "/data/miniconda3_new/envs/vllm-new/lib/python3.10/site-packages/vllm/worker/worker.py", line 285, in execute_worker
    self.cache_engine[virtual_engine].copy(worker_input.blocks_to_copy)
  File "/data/miniconda3_new/envs/vllm-new/lib/python3.10/site-packages/vllm/worker/cache_engine.py", line 102, in copy
    self.attn_backend.copy_blocks(self.gpu_cache, src_to_dsts)
  File "/data/miniconda3_new/envs/vllm-new/lib/python3.10/site-packages/vllm/attention/backends/flashinfer.py", line 56, in copy_blocks
    raise NotImplementedError
NotImplementedError

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/data/miniconda3_new/envs/vllm-new/lib/python3.10/site-packages/uvicorn/protocols/http/httptools_impl.py", line 411, in run_asgi
    result = await app(  # type: ignore[func-returns-value]
  File "/data/miniconda3_new/envs/vllm-new/lib/python3.10/site-packages/uvicorn/middleware/proxy_headers.py", line 69, in __call__
    return await self.app(scope, receive, send)
  File "/data/miniconda3_new/envs/vllm-new/lib/python3.10/site-packages/fastapi/applications.py", line 1054, in __call__
    await super().__call__(scope, receive, send)
  File "/data/miniconda3_new/envs/vllm-new/lib/python3.10/site-packages/starlette/applications.py", line 123, in __call__
    await self.middleware_stack(scope, receive, send)
  File "/data/miniconda3_new/envs/vllm-new/lib/python3.10/site-packages/starlette/middleware/errors.py", line 186, in __call__
    raise exc
  File "/data/miniconda3_new/envs/vllm-new/lib/python3.10/site-packages/starlette/middleware/errors.py", line 164, in __call__
    await self.app(scope, receive, _send)
  File "/data/miniconda3_new/envs/vllm-new/lib/python3.10/site-packages/starlette/middleware/cors.py", line 85, in __call__
    await self.app(scope, receive, send)
  File "/data/miniconda3_new/envs/vllm-new/lib/python3.10/site-packages/starlette/middleware/exceptions.py", line 65, in __call__
    await wrap_app_handling_exceptions(self.app, conn)(scope, receive, send)
  File "/data/miniconda3_new/envs/vllm-new/lib/python3.10/site-packages/starlette/_exception_handler.py", line 64, in wrapped_app
    raise exc
  File "/data/miniconda3_new/envs/vllm-new/lib/python3.10/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
    await app(scope, receive, sender)
  File "/data/miniconda3_new/envs/vllm-new/lib/python3.10/site-packages/starlette/routing.py", line 756, in __call__
    await self.middleware_stack(scope, receive, send)
  File "/data/miniconda3_new/envs/vllm-new/lib/python3.10/site-packages/starlette/routing.py", line 776, in app
    await route.handle(scope, receive, send)
  File "/data/miniconda3_new/envs/vllm-new/lib/python3.10/site-packages/starlette/routing.py", line 297, in handle
    await self.app(scope, receive, send)
  File "/data/miniconda3_new/envs/vllm-new/lib/python3.10/site-packages/starlette/routing.py", line 77, in app
    await wrap_app_handling_exceptions(app, request)(scope, receive, send)
  File "/data/miniconda3_new/envs/vllm-new/lib/python3.10/site-packages/starlette/_exception_handler.py", line 64, in wrapped_app
    raise exc
  File "/data/miniconda3_new/envs/vllm-new/lib/python3.10/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
    await app(scope, receive, sender)
  File "/data/miniconda3_new/envs/vllm-new/lib/python3.10/site-packages/starlette/routing.py", line 72, in app
    response = await func(request)
  File "/data/miniconda3_new/envs/vllm-new/lib/python3.10/site-packages/fastapi/routing.py", line 278, in app
    raw_response = await run_endpoint_function(
  File "/data/miniconda3_new/envs/vllm-new/lib/python3.10/site-packages/fastapi/routing.py", line 191, in run_endpoint_function
    return await dependant.call(**values)
  File "/data/miniconda3_new/envs/vllm-new/lib/python3.10/site-packages/vllm/entrypoints/openai/api_server.py", line 132, in create_chat_completion
    generator = await openai_serving_chat.create_chat_completion(
  File "/data/miniconda3_new/envs/vllm-new/lib/python3.10/site-packages/vllm/entrypoints/openai/serving_chat.py", line 305, in create_chat_completion
    return await self.chat_completion_full_generator(
  File "/data/miniconda3_new/envs/vllm-new/lib/python3.10/site-packages/vllm/entrypoints/openai/serving_chat.py", line 505, in chat_completion_full_generator
    async for res in result_generator:
  File "/data/miniconda3_new/envs/vllm-new/lib/python3.10/site-packages/vllm/engine/async_llm_engine.py", line 746, in generate
    async for output in self._process_request(
  File "/data/miniconda3_new/envs/vllm-new/lib/python3.10/site-packages/vllm/engine/async_llm_engine.py", line 845, in _process_request
    stream = await self.add_request(
  File "/data/miniconda3_new/envs/vllm-new/lib/python3.10/site-packages/vllm/engine/async_llm_engine.py", line 654, in add_request
    self.start_background_loop()
  File "/data/miniconda3_new/envs/vllm-new/lib/python3.10/site-packages/vllm/engine/async_llm_engine.py", line 476, in start_background_loop
    raise AsyncEngineDeadError(
vllm.engine.async_llm_engine.AsyncEngineDeadError: Background loop has errored already.

We have a tracking issue (#5901) for this. Please provide more details there so we can better troubleshoot the underlying cause.

Faced same issue today. Was running a script calling API concurrently and server generated same error after about 15-20k requests

Faced with the same problem, the lowest number of requests was 4000, and the highest number was 45,000.
Two 3080 graphics cards were used to run the glm4-int8 model.


Create a builder that builds a widget that a wraps routes of Navigator.

It is discussion of closed PR https://github.com/flutter/flutter/pull/33706
I suggest add add builder wrapper of navigator routers for access Navigator from above routers of widgets tree.
It is allow  access to Navigator from root widget and control navigation of application. for example I can create abstract Router for Viewmodel or Presenter and control app routing from logic layer without clearly communication with "View" layer.
you say why not wrap MaterialApp in to RootWidget? but in but in this case I will not have access to MediaQuuery, Navigator etc.
you say why you push to navigator RootRoute as first route and control Navigation from him? because netx routes will be have back arrow on AppBar. And I can accidentally pop her from Navigaton. It is no way.
you say why not create custom App with WidgetApp and Navigator? no way, because I want use all features of MaterialApp or CupertionApp which encapsulated in exist Widget.
below is an example;
import 'package:flutter/material.dart';

void main() => runApp(MyApp());

class MyApp extends StatelessWidget {
  @override
  Widget build(BuildContext context) {
    return MaterialApp(
      title: 'Flutter Demo',
      theme: ThemeData(
        primarySwatch: Colors.blue,
      ),
      home: Scaffold(
        appBar: AppBar(),
      ),
      navigatorRoutesWrapperBuilder: (BuildContext context, Widget child) {
        return RootWidget(
          child: child,
        );
      },
    );
  }
}

class RootWidget extends StatefulWidget {
  final Widget child;

  RootWidget({Key key, this.child}) : super(key: key);

  @override
  _RootWidgetState createState() => _RootWidgetState();
}

class _RootWidgetState extends State<RootWidget> {

  RootAppBloc _bloc;

  @override
  void initState() {
    _bloc = RootAppBloc();
    super.initState();

    _bloc.pushLoginScreen.listen((_) {
      Route<dynamic> route = MaterialPageRoute<dynamic>(builder: (BuildContext context) {
        return Text('login');
      });

      Navigator.of(context).pushAndRemoveUntil(route, (Route<dynamic> route) {
        return false;
      });
    });

    _bloc.pushMainScreen.listen((_) {
      // some code
    });
  }

  @override
  Widget build(BuildContext context) {
    return widget.child;
  }
}

class RootAppBloc {
  Stream<void> pushMainScreen;
  Stream<void> pushLoginScreen;
}

The main reason why I like this idea is that with this solution it is possible to put a DialogManager above each route without an extra/custom navigator.
https://medium.com/flutter-community/manager-your-flutter-dialogs-with-a-dialog-manager-1e862529523a

If I am understanding this correctly, this would be the same as Angular's router-outlet concept. We would be able to have a static scaffold that doesn't refresh with every navigation.
If so, would the possibility of having multiple navigators and outlets?

@jsroest your solution on article is not properly work if press back button, it is sadly

It would be nice to have this feature

you can use the https://master-api.flutter.dev/flutter/material/MaterialApp/builder.html
If you want access to the navigator, you can provide the navigator key and access it globally. are there any other use cases that cannot be solved by this?

are there any other use cases that cannot be solved by this?

I have a dialogService that uses the material.showDialog function provided by the Flutter framework. Unfortunately, you can not pass the NavigatorState directly to this function, but you have to pass a context, so it can do the lookup for the navigator itself. This forces me to declare a  dialogService above  every page manually.
The suggested builder would eliminate this repetition.

are there any other use cases that cannot be solved by this?

I have a dialogService that uses the material.showDialog function provided by the Flutter framework. Unfortunately, you can not pass the NavigatorState directly to this function, but you have to pass a context, so it can do the lookup for the navigator itself. This forces me to declare a dialogService above every page manually.
The suggested builder would eliminate this repetition.

I think not able to use navigator state directly in show dialog is a bug. If we enable that do you think that will solve your use case?

Yes, that will solve my use case.

Think web applications

Think web applications

If you meant static content above navigator the Material.builder should work. If you meant nested navigator such as angular router-outlet, we are currently working on that in Navigator 2.0 and router widget

To document this:
If you want to have a "dialog service", put it in your builder:
 MaterialApp(
        navigatorKey: myNavigatorKey,
        builder: (BuildContext context, Widget? child) {
             return MyDialogService(child: child)
},

When you access MyDialogService up in the tree it cannot call showDialog with its own context. But you can use myNavigatorKey.currentContext:
showDialog(context : myNavigatorKey.currentContext, ...)

@chunhtai Quite frankly, I would prefer we could indeed create a builder that builds a widget that a wraps routes of Navigator. This would be more elegant than using keys (I feel every time we use global keys we have a bad architecture). As pointed out above, this could be used to have a static scaffold that doesn't refresh with every navigation. I believe this would also be a nice place to put custom keyboards, that could use the Navigator to close with the back button. They are an example of a necessary "scaffold" that should exist in every page. Where should I put them instead?


Multiple Radio button Group

Hi, your example(type2 and 3) is only applicable for 1 group of radio buttons which sets the index using this.setState on onClick. How can i set the selected index without setting it in this.setState

I find this inflexible, I have to used RadioButton class in order for me to edit the label style.

Thank you for your issue:smile_cat:
I will check and answer soon.

I needed to achieve this functionality too. Here's a project I put together to show off multiple radio button groups on the same page.
https://github.com/swhamilton/react-native-multiple-radio-button-groups
Hope this helps!
@moschan Maybe you could use some of this in the docs for this radio button, seems like a common use case and many people might find it useful.

@swhamilton
Thank you for comment and your example.
I will implement this function in the near future :smile_cat:

I have tried to use two RadioForms on the same component... not working. Both onPress actions are calling the same function :(


SA1649 for enums

Having the following two files:
namespace TestNamespace
{
    public class MySuperClass
    {
    }
}

namespace TestNamespace
{
    public enum TestEnum
    {
        Test = 0
    }
}

SA1649 gets only reported for MySuperClass.
In my stylecop.json I have set enum as topLevelTypes.
Is it not possible to report this warning for enums?

This is currently not supported unfortunately. I'm marking this as duplicate of #2223


[ENHANCEMENT] Forge Support

My immediate next task is getting pickhaxe build forge 1.19.4 working as expected, ideally using Maven to fetch version info.

Closed by 5209a61 and v0.2.0.


[Bugfix][CI/Build][Hardware][AMD] Fix AMD tests, add HF cache, update CK FA, add partially supported model notes

This PR attempts to fix existing failures on AMD tests.

Metrics tests are currently failing because of https://github.com/vllm-project/vllm/pull/6338, which accidentally added a hard requirement for vllm_flash_attn in vllm/spec_decode/draft_model_runner.py. This is not installed on ROCm, so the correct ROCm FA component is used instead.
Basic correctness tests are currently failing because the test harness unconditionally installs flashinfer in a way that is not supported on ROCm. This installation is made optional: this shouldn't affect the esoteric case where flashinfer isn't correctly installed during CUDA tests, because 1. this will give an obvious error message at the beginning of the test and 2. flashinfer ModuleNotFoundError is also pretty obvious.
Entrypoints tests are failing because they now depend on peft (due to https://github.com/vllm-project/vllm/pull/4645) which is not currently installed in the AMD container. This in turns wants updated botocore and hence boto3 and awscli. These requirements are included in requirements-test.txt but are a small subset of it, so we do not install it for now. Once more AMD tests are enabled, we will begin using requirements-test.txt in earnest.
Basic correctness tests were also failing because https://github.com/vllm-project/vllm/pull/6549 erroneously introduced a test involving compressed tensors quant on ROCm, where it is not supported.
Model download sometimes takes an inordinately long time due to a flaky network configuration and contributes to tests (typically Distributed and Vision Language) hitting the 90 min timeout. This PR adds a HuggingFace models cache on AMD machines similar to what was done in https://github.com/vllm-project/buildkite-ci/pull/8. Note that this might not allow flaky tests to pass immediately, but the hope is that once each machine has had a chance to run all tests and incrementally build up its cache, the issue will eventually be resolved.
Vision Language Models and Distributed tests are also failing because ROCm's Triton FA kernel hangs on a variety of cases, including PaliGemma full-precision tests (added in https://github.com/vllm-project/vllm/pull/5189) and multimodal broadcast with Phi-3-Vision. This is due to issues during Triton compilation which is solved by updating Triton.
Even after updating Triton, Phi-3-Vision and PaliGemma are still problematic under certain settings because too much shared memory is reserved by the Triton kernel on some devices. As a temporary workaround, CK FA is updated and used instead for the relevant tests and the associated warnings for partial model support in ROCm are added.

 PR Checklist (Click to Expand)
Thank you for your contribution to vLLM! Before submitting the pull request, please ensure the PR meets the following criteria. This helps vLLM maintain the code quality and improve the efficiency of the review process.
PR Title and Classification
Only specific types of PRs will be reviewed. The PR title is prefixed appropriately to indicate the type of change. Please use one of the following:

[Bugfix] for bug fixes.
[CI/Build] for build or continuous integration improvements.
[Doc] for documentation fixes and improvements.
[Model] for adding a new model or improving an existing model. Model name should appear in the title.
[Frontend] For changes on the vLLM frontend (e.g., OpenAI API server, LLM class, etc.)
[Kernel] for changes affecting CUDA kernels or other compute kernels.
[Core] for changes in the core vLLM logic (e.g., LLMEngine, AsyncLLMEngine, Scheduler, etc.)
[Hardware][Vendor] for hardware-specific changes. Vendor name should appear in the prefix (e.g., [Hardware][AMD]).
[Misc] for PRs that do not fit the above categories. Please use this sparingly.

Note: If the PR spans more than one category, please include all relevant prefixes.
Code Quality
The PR need to meet the following code quality standards:

We adhere to Google Python style guide and Google C++ style guide.
Pass all linter checks. Please use format.sh to format your code.
The code need to be well-documented to ensure future contributors can easily understand the code.
Include sufficient tests to ensure the project to stay correct and robust. This includes both unit tests and integration tests.
Please add documentation to docs/source/ if the PR modifies the user-facing behaviors of vLLM. It helps vLLM user understand and utilize the new features or changes.

Notes for Large Changes
Please keep the changes as concise as possible. For major architectural changes (>500 LOC excluding kernel/data/config/test), we would expect a GitHub issue (RFC) discussing the technical design and justification. Otherwise, we will tag it with rfc-required and might not go through the PR.
What to Expect for the Reviews
The goal of the vLLM team is to be a transparent reviewing machine. We would like to make the review process transparent and efficient and make sure no contributor feel confused or frustrated. However, the vLLM team is small, so we need to prioritize some PRs over others. Here is what you can expect from the review process:

 After the PR is submitted, the PR will be assigned to a reviewer. Every reviewer will pick up the PRs based on their expertise and availability.
 After the PR is assigned, the reviewer will provide status update every 2-3 days. If the PR is not reviewed within 7 days, please feel free to ping the reviewer or the vLLM team.
 After the review, the reviewer will put an  action-required label on the PR if there are changes required. The contributor should address the comments and ping the reviewer to re-review the PR.
 Please respond to all comments within a reasonable time frame. If a comment isn't clear or you disagree with a suggestion, feel free to ask for clarification or discuss the suggestion.

Thank You
 Finally, thank you for taking the time to read these guidelines and for your interest in contributing to vLLM. Your contributions make vLLM a great tool for everyone!

/ready

This in turns wants updated botocore and hence boto3 and awscli

How is peft related to boto3 and awscli? Could you explain more?

This in turns wants updated botocore and hence boto3 and awscli

How is peft related to boto3 and awscli? Could you explain more?

It's linked via urllib3 to botocore, which is linked to awscli and boto3. As far as I can tell, on the base image we use, awscli boto3 botocore peft is the smallest set to use to get the related dependencies working and reenable any tests we currently support.

Rebased, and is all green except LM Eval Small Models which seems to be broken in upstream. Ready to ship!


WSL 2 Clipboard issue

Feature description
Currently win32yank is still used, but officials' wiki seem to have proposed a new method.
Ref: https://github.com/neovim/neovim/wiki/FAQ#how-to-use-the-windows-clipboard-from-wsl
Additional information
No response

looks promising! have u tried it yet?

Tried it, the paste operation requires more time to be finished compared to win32yank.
I guess it have to do with my powershell need some time to load env.


Make MaintainedModel thread-safe

FEATURE REQUEST
Inspiration
The current MaintainedModel implementation uses global variables to change the behavior of auto-updates during data loads.  Depending on the apache configuration, those globals could be shared not only among threads, but even between "processes" (think mod_perl, where it's all really one process and each request uses the same memory). We currently don't have any issue because we don't have an edit interface.  The only time .save() is ever called on any process under apache is during data validation under apache or on the command line during load.  But, if we add an edit interface (even just for admins), if an edit occurs during a user running a data validation, the save() behavior could become unpredictable if Apache is configured in a certain way.
Description
Make MaintainedModel thread-safe.  I have looked into this and I have a plan.  Right now, it's just in the form of rough notes, which I will put in the comment section below, but the amount of effort will be minimal.  The crux of it is that I will use thread.local() to prevent different threads/processes from interfering with one another.  I'm also going to use this opportunity to make some general improvements.  Some of these plans may overlap with 1 or 2 other existing issues and I will mrk them as duplicates if that happens.
Alternatives
Tweak the apache config to keep data between threads separate.
Dependencies
None
Comment
MaintainedModel improvements over using globals...

PR #735

[x] Eliminate global variables (make them class attributes)
[x] Move the setters/getters for the variables into the class as @classmethods
[x] Move the decorators into the class as @staticmethods

PR yet to be submitted

[x] use thread.local() to make the class attributes thread-local. This should allow the "global" variables to be "thread-safe"

https://docs.python.org/3/library/threading.html#thread-local-data
Pay attention to: https://bugs.python.org/issue24020
I discovered this strategy via this stack post: https://stackoverflow.com/questions/13171860/how-django-handles-simultaneous-requests-with-concurrency-over-global-variables
~Not sure if globals() is a viable alternative.  I don't yet understand what it's for: https://docs.python.org/3/library/functions.html#globals~

PR yet to be submitted

Create 2 more function decorators to simplify things like loading scripts:

[x] defer_autoupdates

[x] handles the call to perform_buffered_updates (but checks to see if autoupdates are completely disabled first)
[x] handles the setting of disable_autoupdates, etc
[x] catches exceptions to restore the "global variables" and re-raises
[x] restores the "global variables" after function runs

[x] no_autoupdates

[x] disables both buffering and autoupdates
[x] ensures mutually exclusive with defer_autoupdates
[x] restores the "global variables" after function runs

PR yet to be submitted

[ ] Add a fixture to all tests that clears the buffer and sets "global variables" to default

https://stackoverflow.com/questions/73865462/reset-global-variables-between-two-tests-on-pytest

PR yet to be submitted

[ ] Add the ability for setter methods to be checked that they do not use a maintained field to set their value

ISSUE OWNER SECTION
Assumptions

List of assumptions that the code will not explicitly address/check
E.g. We will assume input is correct (explaining why there is no validation)

Limitations

A list of things this work will specifically not do
E.g. This feature will only handle the most frequent use case X

Affected Components

change: File path or DB table ...
add: Environment variable or server setting
delete: External executable or cron job

Requirements

[ ] 1. List of numbered conditions to be met for the feature
[ ] 2. E.g. Every column/row must display a value, i.e. cannot be empty
[ ] 3. Numbers for reference & checkboxes for progress tracking

DESIGN
Interface Change description

None provided
Code Change Description

None provided
Tests

[ ] 1. A description of at least one test for each requirement above.
[ ] 2. E.g. Test for req 2 that there's an exception when display value is ''
[ ] 3. Numbers for reference & checkboxes for progress tracking

I'm currently on vacation, but Zoe is at work and my niece and nephew are still asleep, so I'm doing some "vanity coding".  I want to eventually publish MaintainedModel, so I'm working on this issue to make MaintainedModel more robust and work in other environments.

Notes, so that I can pick up where I left off in the office...
Add the ability for setter methods to be checked that they do not use a maintained field to set their value.
I still have to:

[ ] update all the tests
[ ] update comments and error messages to not reference deleted methods
[x] Convert the rebuild script
[x] create a disable_autoupdates decorator and apply it to the validation interface.

[x] maybe figure out a way to make this disable defer wrappers it encounters in a sub-context

[x] I need to check files that DO NOT import MaintainedModel, but make calls to MaintainedModel methods off of the model objects
[x] I decided that I should obtain the current coordinator anytime I need it.  I decided this when debugging test test_get_name_triggers_autoupdate.  It was creating a disabled coordinator in a context manager and then after that context exited, I wanted to call get_name off the model object to lazy-update the database, but the popped coordinator retained in the instance was still disabled, so it was not auto-updating.  That was the behavior I'd intended, but when working with the code, if baffled me for a few minutes and I only understood what was happening by issuing some debug prints, so I no longer think that the coordinator should be retained in the MaintainedModel instance.  It should retrieve the current coordinator each time with MaintainedModel.get_coordinator().

Regarding lazy_update, I was looking for a way to do it implicitly, and I noted that I don't want to do it in init, because init happens when saving AND querying, so I realized that I need perhaps to find out when a QuerySet iterable is instantiating a Model object.  QuerySet calls Model.from_db() (a class method) when the model is instantiated.  My hope is that models are only instantiated by QuerySet when they are rendered, and that a QuerySet is only used for selects.

[ ] Override Model.from_db to perform lazy updates implicitly

https://docs.djangoproject.com/en/4.2/ref/models/instances/#django.db.models.Model.from_db
YES - from_db is only called as the QuerySet iterable iterates, so if I override that method, I should be able to lazy-update implicitly!  See the response to my comment on this answer:
https://stackoverflow.com/a/70521157/2057516

This is done/merged.


make auto-update of nextls configurable

nextls is automatically fetched for more common OS & platforms, but for other
platforms it should be possible to disable the auto-update, or at least
handle it more gracefully than just bailing out.
Currently I'm just building nextls locally in a repo and starting it listening
on the appropriate port as required.
What's a reasonable approach here? I see a few options:

provide a way to skip auto-update but still have nextls usable
handle download failure more gracefully with:

bail with a reasonable error message if not MacOS or Linux
ditto for other architectures
check if curl is available
handle failure of curl itself

If you set the cmd field to the location of your locally built exe, it will disable auto update and still run it through stdio as usual.

Also do you mean auto update or the auto install?
Auto update is done by next ls (updates itself in place) and auto install is when the nvim plugin downloads it if it can't find one and cmd isn't set

Also what platform are you on?


Loading more than one tile async

Hey,
I did a PR while ago when the lib was still in version 1.something to fetch more than one tile at the same time. And it seems that this code was not introduced in version 2.0, all tiles are loaded in sequence (takes too long to load all tiles if they're not as a resource inside the app).
Do you have interest in updating your lib to load more than one tile at the same time? I could possibly work on that and create another PR but before I would like to know why do you remove the code I did for this (was there a bug) not in the scope of the lib?
Thanks,

Hi - I remember the update.  I don't remember specifically how the final threading model was decided on, but virtually everything in version 2 was reviewed and rewritten.  I would guess it's that the current iteration uses a ThreadPoolExecutor, so it's not really one at a time, it's just ordered - I believe that it'll start as many operations as there are cores available, but yes they'd be executed in order (and might potentially block subsequent ops; I'd have to look it up and am only half-in right now (still at work)).
Do you believe your approach would demonstrat better actual (versus perceptual) performance?  Can you briefly describe (or remind me) what your alternate approach was?  Unfortunately I squashed and lost history with v2.

Hey,
I'm just getting familiar with your whole new code. But it seems that is exactly the same point as the previous versoin. You have a ThreadPoolExecutor that usually only have one async task which is responsible for loading all tiles sequentially (TileRenderTask in TileCanvasViewGroup.java).
A faster approach would be having each tile render in a separate Thread this way the ThreadPoolExecutor could run more than one Thread at the same time. But in a approach like that we need to have a better way to decide if the task should run or not, because every time you scroll or do something else you should cancel all previous queued tasks and queue new tasks for each tile (you just need to cancel one task in your code right now and create a new instance of TileRenderTask).
I worked in this multi thread approach because I'm using the TileView with assets on the Internet, being able to download more than one at the same time is a must. I could possibly work in a way to make Tile loading multi threaded and do a PR for you to review.
It seems that my fork is still in the same state as the PR I did, if you want to take a look
https://github.com/bnsantos/TileView

Thanks for the link - that does look familiar.
The AsyncTask for fetching tile bitmaps in v1 was not using executeOnExecutor(THREAD_POOL_EXECUTOR), but v2 does.
AFAICT, using AsyncTask and passing in the static ThreadPoolExecutor will use multiple threads - one per available core (see the third link)
https://github.com/android/platform_frameworks_base/blob/master/core/java/android/os/AsyncTask.java#L202
Am I'm misreading that?

Hey Mike,
You're not wrong about the ThreadPoolExecutor API. If you have more than one Tasks/Thread it will be able to run multiple tasks at the same time depending on the number of the cores available. But in v2 you have only one TileRenderTask per time in the ThreadPoolExecutor. This way doesn't matter if you have 1, 2 or N cores ThreadPoolExecutor will have only one Task/Thread.
Every time the user stops span, zomming or first open the TileView the TileRenderTask is responsible for loading all tiles in a sequence (only starts to load/render a tile after the previous tile was successfully loaded rendered). And that TileRenderTask goes to ThreadPoolExecutor and lies there until either it finishes or some zoom or span happens. In case of zoom or span happens you add a new TileRenderTask to load the tile in the new position while the previous task is being invalidate (it will be removed eventually from ThreadPoolExecutor not sure when exactly) there will be more than one Task in the ThreadPoolExecutor (only time that will happen).
To properly use MultiThread you need to spawn a lot of Threads/AsyncTask in the ThreadPoolExecutor. So it can run more than one at a time depending on the number of cores available.
To make it more clear I recorded two videos of my app using TileView. One with the version v1 and the other with the version v2. The images come from a website and they were not cached previously. I ran in the same emulator both versions.
Tile Loading v1 - MultiThread
Tile Loading v2 - SingleThread
Even though the Tiles are loaded in sequence for v1 and v2 you can see how fast v1 is. It is fast because it is downloading more than one image at the same time because I spawn a TileDownloadRunner for every Tile.
I hope you understand my points

makes perfect sense, i'm embarrassed to have not known that - yes, by all means PR - or if you don't have the time, i can do it based on your original work - let me know what works best for you.
if you do decide to create the update yourself, please try to match style and convention (naming, whitespace, class structure, etc) as close to existing as possible.
thanks again - i appreciate your work and cluing me in onto what i'd misunderstood

Good,
I'll work on those changes and keep the same style convention!
Thanks

Hi bnsantos,
Are you actually working on this? I would like to give it a try, but if you have already started, i guess there is no point doing the same thing twice ;)

Hey  @peterLaurence ,
No I didn't even started this, I was thinking of starting this right now. Did you start already?
Sorry for the delayed reply I was pulled into some urgent stuff here and I didn't even have time to read your message.

Hi @bnsantos ,
No problem. I planned to do it but i don't have time right now. As you may have seen, some problem have been fixed the last couple of weeks, including a critical one (see #260).
Look forward to see what you hearing whay you come up with.  If you make it work with tileView.shouldRenderWhilePaning( true ), it would be perfect.

Hey @peterLaurence
Just finished a "working" version in my fork if you could please take a look because you're more up to date on those issues. I'm still trying to catch up the issues we're open while I was absent.
Also added an example to download tiles from internet.

Ok i will try that!

Looks very good! It works with shouldRenderWhilePaning( true ) (LargeImage example).
And your example works nice too.

Will take a closer look at your code as soon as i have time. (i hope tomorrow). In the meantime, you may have to comply with Mike's code style :)

@bnsantos yes please try to conform to existing code style, whitespace, naming conventions, patterns, etc.  if you could re-commit when that's done and post back, i/we can do a quick code review and try to get this integrated asap
thanks again

So i did some checking. Even if the overall performance of this new implementation of tile loading is very good, i was curious to know if it is better than the old one for local tiles (ofc this new implementation is better for tiles on the net). So basically i defined this in TileView to log the render time :
private long time;

@Override
public void onRenderStart() {
  time = System.currentTimeMillis();
}

@Override
public void onRenderComplete() {
  long delta = System.currentTimeMillis() - time;
  System.out.println( "delta : " + delta );
}

Then i picked the LargeImage demo example and i double-taped several times to go through all DetailLevel in a cyclic manner. On my nexus 5 (2013 edition), those are delta with the old implementation :
02-12 23:28:37.120 11590-11590/tileview.demo I/System.out: delta : 196
02-12 23:28:39.514 11590-11590/tileview.demo I/System.out: delta : 1
02-12 23:28:39.580 11590-11590/tileview.demo I/System.out: delta : 0
02-12 23:28:41.555 11590-11590/tileview.demo I/System.out: delta : 267
02-12 23:28:42.694 11590-11590/tileview.demo I/System.out: delta : 1
02-12 23:28:42.705 11590-11590/tileview.demo I/System.out: delta : 1
02-12 23:28:42.721 11590-11590/tileview.demo I/System.out: delta : 0
02-12 23:28:42.759 11590-11590/tileview.demo I/System.out: delta : 0
02-12 23:28:43.312 11590-11590/tileview.demo I/System.out: delta : 230
02-12 23:28:47.266 11590-11590/tileview.demo I/System.out: delta : 1
02-12 23:28:47.310 11590-11590/tileview.demo I/System.out: delta : 1
02-12 23:28:50.015 11590-11590/tileview.demo I/System.out: delta : 133
02-12 23:29:00.913 11590-11590/tileview.demo I/System.out: delta : 176
I believe the 0 and the 1 are caused by a requestRender directly followed by a cancelRender, which happens at a DetailLevel change.
And with the new implementation (one thread per tile) :
02-12 23:21:05.947 6090-6090/tileview.demo I/System.out: delta : 135
02-12 23:21:07.155 6090-6090/tileview.demo I/System.out: delta : 164
02-12 23:21:10.288 6090-6090/tileview.demo I/System.out: delta : 229
02-12 23:21:10.473 6090-6090/tileview.demo I/System.out: delta : 138
02-12 23:21:12.557 6090-6090/tileview.demo I/System.out: delta : 196
02-12 23:21:16.723 6090-6090/tileview.demo I/System.out: delta : 214
02-12 23:21:16.886 6090-6090/tileview.demo I/System.out: delta : 101
02-12 23:21:18.753 6090-6090/tileview.demo I/System.out: delta : 146
02-12 23:21:21.516 6090-6090/tileview.demo I/System.out: delta : 237
02-12 23:21:21.719 6090-6090/tileview.demo I/System.out: delta : 157
In conclusion render times are very close, perhaps slightly better for the new method. Maybe you can test with your own devices.

@bnsantos
About your code, while i like how the Runnable post their result on the handler which runs on ui thread, it seems like you assume that the last TileRenderRunnable you create in TileRenderPoolExecutor.queue will be the last to end, thus it calls TileCanvasViewGroup.onRenderTaskPostExecute. It seems to behave like that, but if the last TileRenderRunnable finishes before one of the other launched before, the call of TileCanvasViewGroup.onRenderTaskPostExecute which then calls a TileCanvasViewGroup.cleanup. That last call a little too early may cause an issue but i am not sure of that.

@moagrius
I'm aware of your new coding style because of that I already refactored a lot of think, and before the PR I would ask you to review again.
@peterLaurence
The render time should be the same because I didn't change the BitmapProvider implementation (in the example you mentioned BitmapProviderAssets) what I'm doing is a way to render more than one Tile per time if the device has more than one core (can even load 3, 4, ... 8 at the same time). So I would suggest you to add another log
@Override
public void onRenderStart() {
  time = System.currentTimeMillis();
  System.out.println( "Start loading tile : " + <some tile identifier here> );
}

@Override
public void onRenderComplete() {
  long delta = System.currentTimeMillis() - time;
  System.out.println( "Finished rendering tile: "+ <some tile identifier here>  + " - delta : " + delta );
}

Then you could see if more than one tile is rendering at the same time.
Now regarding the last TileRenderRunnable being queued that I add a flag saying that this is the last one to be queued but not necessary it will be the last one to be executed.  I missed that . Going to take a look to see how can I fix that to make sure to call TileCanvasViewGroup.onRenderTaskPostExecute after all queued TileRenderRunnable are executed. Thanks

I'm aware of your new coding style because of that I already refactored a lot of think, and before the PR I would ask you to review again.

:thumbsup:

@bnsantos
I know that the same BitmapProvider used by each thread is the same as before. But the whole task of running threads versus one thread loading all tiles can differ in execution time, depending on how concurrency is handled at lower levels and how the overhead of switching between threads affects performance.
For tiles on a network, thread are mostly IO-bound so having several threads is definitely better (maybe more than the number of cores could be better).
For local tiles, we are mostly CPU-bound, and then having a thread equal to number of cores is good compromise.
As for the test i made, i'd like to clarify that TileView.onRenderStart() is called once before the first tile render, and TileView.onRenderComplete() is called after the last tile render. So the delta between the two calls should measure the execution time of the whole rendering task. That's what i wanted to benchmark.
That's reassuring though that those execution time are equivalent.

@peterLaurence
My bad, got confused between Tile and TileView. You're right about the loading time. And no worries with switching threads because according the [documentation] (https://developer.android.com/reference/java/util/concurrent/ThreadPoolExecutor.html)

Thread pools address two different problems: they usually provide improved performance when executing large numbers of asynchronous tasks, due to reduced per-task invocation overhead, and they provide a means of bounding and managing the resources, including threads, consumed when executing a collection of tasks. Each ThreadPoolExecutor also maintains some basic statistics, such as the number of completed tasks.

So I think it's finally done, now I have a custom implementation of ThreadPoolExecutor that has a hooker method afterExecute that checks if it is the last running thread.

@bnsantos
:+1: Looks good to me

if i'm looking at the right branch (master) looks like there's still some style issues (mostly whitespace: https://github.com/moagrius/TileView/compare/master...bnsantos:master)...  did you want to PR and we can do line comments, or do you want to fix it up a bit first?

@moagrius
So I changed more coding style stuff. Now I'll create the PR and we can do line comments.
Thanks


QUICLocalFlowController doesn't create MAX(_STREM)_DATA frame

Connection level local Flow Controller doesn't create MAX_DATA frame.

Conditions

Set initial_max_data 8192
Transfer 64KB file

Server Side Debug Log

[Aug 31 15:31:49.677] {0xb000c000} DEBUG: <QUICNetVConnection.cc:1472 (_init_flow_c> (quic_flow_ctrl)                 [b7c75d8d-4d307134] [REMOTE] 0/8192
[Aug 31 15:31:49.741] {0xb000c000} DEBUG: <QUICNetVConnection.cc:1284 (_packetize_f> (quic_flow_ctrl)                 [b7c75d8d-4d307134] [REMOTE] 1142/8192
[Aug 31 15:31:49.741] {0xb000c000} DEBUG: <QUICNetVConnection.cc:1284 (_packetize_f> (quic_flow_ctrl)                 [b7c75d8d-4d307134] [REMOTE] 2284/8192
[Aug 31 15:31:49.741] {0xb000c000} DEBUG: <QUICNetVConnection.cc:1284 (_packetize_f> (quic_flow_ctrl)                 [b7c75d8d-4d307134] [REMOTE] 3426/8192
[Aug 31 15:31:49.742] {0xb000c000} DEBUG: <QUICNetVConnection.cc:1284 (_packetize_f> (quic_flow_ctrl)                 [b7c75d8d-4d307134] [REMOTE] 4568/8192
[Aug 31 15:31:49.742] {0xb000c000} DEBUG: <QUICNetVConnection.cc:1284 (_packetize_f> (quic_flow_ctrl)                 [b7c75d8d-4d307134] [REMOTE] 5710/8192
[Aug 31 15:31:49.742] {0xb000c000} DEBUG: <QUICNetVConnection.cc:1284 (_packetize_f> (quic_flow_ctrl)                 [b7c75d8d-4d307134] [REMOTE] 6852/8192
[Aug 31 15:31:49.742] {0xb000c000} DEBUG: <QUICNetVConnection.cc:1284 (_packetize_f> (quic_flow_ctrl)                 [b7c75d8d-4d307134] [REMOTE] 7898/8192
[Aug 31 15:31:49.742] {0xb000c000} DEBUG: <QUICNetVConnection.cc:1284 (_packetize_f> (quic_flow_ctrl)                 [b7c75d8d-4d307134] [REMOTE] 7988/8192
[Aug 31 15:31:49.743] {0xb000c000} DEBUG: <QUICNetVConnection.cc:1284 (_packetize_f> (quic_flow_ctrl)                 [b7c75d8d-4d307134] [REMOTE] 8192/8192

Client Side Debug Log
QUICFlowController::_threshold is added in the debug log. Format is offset/threshold/limit.

[Aug 31 15:31:49.610] {0x10aa13000} DEBUG: <QUICNetVConnection.cc:1470 (_init_flow_c> (quic_flow_ctrl)                 [4d307134-b7c75d8d] [LOCAL] 0/8192
[Aug 31 15:31:49.819] {0x10aa13000} DEBUG: <QUICNetVConnection.cc:1388 (_recv_and_ac> (quic_flow_ctrl)                 [4d307134-b7c75d8d] [LOCAL] 1142/1024/8192
[Aug 31 15:31:49.820] {0x10aa13000} DEBUG: <QUICNetVConnection.cc:1396 (_recv_and_ac> (quic_flow_ctrl)                 [4d307134-b7c75d8d] [LOCAL] 1142/807/9334
[Aug 31 15:31:49.825] {0x10aa13000} DEBUG: <QUICNetVConnection.cc:1388 (_recv_and_ac> (quic_flow_ctrl)                 [4d307134-b7c75d8d] [LOCAL] 2284/807/9334
[Aug 31 15:31:49.825] {0x10aa13000} DEBUG: <QUICNetVConnection.cc:1396 (_recv_and_ac> (quic_flow_ctrl)                 [4d307134-b7c75d8d] [LOCAL] 2284/1615/10476
[Aug 31 15:31:49.826] {0x10aa13000} DEBUG: <QUICNetVConnection.cc:1388 (_recv_and_ac> (quic_flow_ctrl)                 [4d307134-b7c75d8d] [LOCAL] 3426/1615/10476
[Aug 31 15:31:49.826] {0x10aa13000} DEBUG: <QUICNetVConnection.cc:1396 (_recv_and_ac> (quic_flow_ctrl)                 [4d307134-b7c75d8d] [LOCAL] 3426/2422/11618
[Aug 31 15:31:49.827] {0x10aa13000} DEBUG: <QUICNetVConnection.cc:1388 (_recv_and_ac> (quic_flow_ctrl)                 [4d307134-b7c75d8d] [LOCAL] 4568/2422/11618
[Aug 31 15:31:49.827] {0x10aa13000} DEBUG: <QUICNetVConnection.cc:1396 (_recv_and_ac> (quic_flow_ctrl)                 [4d307134-b7c75d8d] [LOCAL] 4568/3230/12760
[Aug 31 15:31:49.827] {0x10aa13000} DEBUG: <QUICNetVConnection.cc:1388 (_recv_and_ac> (quic_flow_ctrl)                 [4d307134-b7c75d8d] [LOCAL] 5710/3230/12760
[Aug 31 15:31:49.827] {0x10aa13000} DEBUG: <QUICNetVConnection.cc:1396 (_recv_and_ac> (quic_flow_ctrl)                 [4d307134-b7c75d8d] [LOCAL] 5710/4038/13902
[Aug 31 15:31:49.828] {0x10aa13000} DEBUG: <QUICNetVConnection.cc:1388 (_recv_and_ac> (quic_flow_ctrl)                 [4d307134-b7c75d8d] [LOCAL] 6852/4038/13902
[Aug 31 15:31:49.828] {0x10aa13000} DEBUG: <QUICNetVConnection.cc:1396 (_recv_and_ac> (quic_flow_ctrl)                 [4d307134-b7c75d8d] [LOCAL] 6852/4845/15044
[Aug 31 15:31:49.829] {0x10aa13000} DEBUG: <QUICNetVConnection.cc:1388 (_recv_and_ac> (quic_flow_ctrl)                 [4d307134-b7c75d8d] [LOCAL] 7988/4845/15044
[Aug 31 15:31:49.829] {0x10aa13000} DEBUG: <QUICNetVConnection.cc:1396 (_recv_and_ac> (quic_flow_ctrl)                 [4d307134-b7c75d8d] [LOCAL] 7988/5649/16180
[Aug 31 15:31:49.829] {0x10aa13000} DEBUG: <QUICNetVConnection.cc:1388 (_recv_and_ac> (quic_flow_ctrl)                 [4d307134-b7c75d8d] [LOCAL] 8192/5649/16180
[Aug 31 15:31:49.829] {0x10aa13000} DEBUG: <QUICNetVConnection.cc:1396 (_recv_and_ac> (quic_flow_ctrl)                 [4d307134-b7c75d8d] [LOCAL] 8192/5793/16384

offset + threshold is always smaller than limit. Which means MAX_DATA is never created.
https://github.com/apache/trafficserver/blob/01030555a4cfe8e421886673e8dfdc592e13c53a/iocore/net/quic/QUICFlowController.cc#L165-L167
On stream level, I haven't seen this. But this could be happen when the QUICRateAnalyzer::_rate get low.

Fixed by 75a00c564f11e59a5145e1f9a0c1332770fd11bf


Add usage of createHref in README

As a beginner, it is really confusing how to construct links in the DOM. The example in the README is incomplete without a working example of links. I have tried using createHref after doing a lot of searching, but it is not working and I am getting an error like this:
TypeError: Cannot read property '_namespace' of undefined
    at RouterSource.createHref (VM986 bundle.js:3568)
    at HomeComponent (VM986 bundle.js:3432)
    at MapOp.f (VM986 bundle.js:3562)
    at _try (VM986 bundle.js:17778)

My example source code:
import xs from 'xstream';
import { run } from '@cycle/run';
import { makeDOMDriver, div, p, a } from '@cycle/dom';
import { routerify } from 'cyclic-router';
import { makeHistoryDriver } from '@cycle/history';
import switchPath from 'switch-path';

function HomeComponent(sources) {
  const { router: { createHref }} = sources;
  const aboutLink = createHref('/about');
  const vdom$ = xs.of(false).map(() => div([
    p('Home page'),
    a({ attrs: { href: aboutLink }}, 'About')
  ]));

  return {
    DOM: vdom$
  }
}

function AboutComponent(sources) {
  const vdom$ = xs.of(false).map(() => div([
    p('About page')
  ]));

  return {
    DOM: vdom$
  }
}

function main(sources) {
  const pageSink$ = sources.router.routedComponent({
    '/': HomeComponent,
    '/about': AboutComponent
  })(sources);

  return {
    DOM: pageSink$.map(c => c.DOM).flatten(),
    router: xs.of('/')
  };
}

const mainWithRouting = routerify(main, switchPath);

run(mainWithRouting, {
  DOM: makeDOMDriver('#app'),
  history: makeHistoryDriver()
});

Examples exist in the tests, like here.  createHref only works if you update the router source for each component like so component({...sources, router: router.path('/about')})

I am looking for something like this
https://github.com/nakaji-dayo/cyclic-router/tree/master/example
But that is using an older version of cyclic-router I guess, the example need not to be as exhaustive like the above, just a simple one to explain how to use createHref from the main page or the components.


Build(deps): Bump @patternfly/react-charts from 6.51.5 to 6.51.19 (#3124)

Pull request opened by the merge tool on behalf of #3124

Merged in f0af17c4f53a85dca993b634539093af101863ad


Added required markdown files, rendered them in nextjs.

Fixes issue

Fixes #273
Changes proposed

I added 5 markdown files namely quickstart.md, getting-started.md, troubleshooting.md, faq.md and header.md.
Next, I added next.config.js that enables MDX support.
Added rermark-html.js that transforms markdown files into HTML using remark.
Added api.js inside a new folder lib/ to fetch the markdown files from the components folder and convert them into HTML.
Lastly, added index.js inside pages/docs/ that renders the documentation.

Check List (Check all the applicable boxes)

[x] My code follows the code style of this project.
[x] My change requires changes to the documentation.
[x] I have updated the documentation accordingly.
[x] This PR does not contain plagiarized content.
[x] The title of my pull request is a short description of the requested changes.

Screenshots

Note to reviewers

I was not able to run the project locally due to issues with my local machine. However, the changes might be significant for the further development of the project.

@SaptarshiSarkar12 yes sure, on it 👍

@markandey1414 Can you please delete the packages published on your fork repo? You can read more here to know how to delete an entire package.

@SaptarshiSarkar12 yes deleted

Thank you @markandey1414.
@markandey1414 How much is done? Also, I think that you are making changes in the project root, but, the website source code is present in the Website directory. Please make changes there.

@SaptarshiSarkar12 Yes. I've been trying to wrap my head around this, have issues working with app router.

@SaptarshiSarkar12 Yes, I checked and added the required code. Have a look.

@markandey1414 Please resolve the deployment failures.

@markandey1414 Here are the deployment logs provided by Vercel.
[22:08:18.814] ./app/docs/doc.js
[22:08:18.814] 19:22  Error: Do not pass children as props. Instead, nest children between the opening and closing tags.  react/no-children-prop
[22:08:18.814] 22:24  Error: Do not pass children as props. Instead, nest children between the opening and closing tags.  react/no-children-prop
[22:08:18.815] 23:24  Error: Do not pass children as props. Instead, nest children between the opening and closing tags.  react/no-children-prop
[22:08:18.815] 24:24  Error: Do not pass children as props. Instead, nest children between the opening and closing tags.  react/no-children-prop
[22:08:18.815] 25:24  Error: Do not pass children as props. Instead, nest children between the opening and closing tags.  react/no-children-prop
[22:08:18.815]
[22:08:18.815] info  - Need to disable some ESLint rules? Learn more here: https://nextjs.org/docs/basic-features/eslint#disabling-rules
[22:08:18.845] Error: Command "npm run build" exited with 1

The index page (the main page) of docs should be named page.js. I think you need to rename docs.js to page.js.

@SaptarshiSarkar12 but wouldn't that lead to conflict? I've added it similar to about page.

@markandey1414 No, as the page.js file will be present in docs directory.

@markandey1414 Please fix the deployment failure. The logs from Vercel are -
[22:39:12.800] Failed to compile.
[22:39:12.801]
[22:39:12.801] ./app/docs/page.js
[22:39:12.801] ReactServerComponentsError:
[22:39:12.801]
[22:39:12.801] You have a Server Component that imports next/router. Use next/navigation instead.
[22:39:12.801] Learn more: https://nextjs.org/docs/app/api-reference/functions/use-router
[22:39:12.801]
[22:39:12.801] Import trace:
[22:39:12.801]   ./app/docs/page.js
[22:39:12.801]
[22:39:12.802]
[22:39:12.802] > Build failed because of webpack errors
[22:39:12.827] Error: Command "npm run build" exited with 1

@markandey1414 Please fix the deployment failure. Here are the logs from Vercel.
[22:47:33.859] Failed to compile.
[22:47:33.859]
[22:47:33.860] ./app/docs/page.js
[22:47:33.860] ReactServerComponentsError:
[22:47:33.860]
[22:47:33.860] "getStaticProps" is not supported in app/. Read more: https://nextjs.org/docs/app/building-your-application/data-fetching
[22:47:33.860]
[22:47:33.860]     ,-[/vercel/path0/Website/app/docs/page.js:23:1]
[22:47:33.860]  23 |   )
[22:47:33.860]  24 | }
[22:47:33.860]  25 |
[22:47:33.860]  26 | export async function getStaticProps() {
[22:47:33.860]     :                       ^^^^^^^^^^^^^^
[22:47:33.860]  27 |   const headerHtml = await getComponentHtml('header')
[22:47:33.860]  28 |   const quickstartHtml = await getComponentHtml('quickstart')
[22:47:33.861]  29 |   const gettingStartedHtml = await getComponentHtml('getting-started')
[22:47:33.861]     `----
[22:47:33.861]
[22:47:33.861] File path:
[22:47:33.861]   ./app/docs/page.js
[22:47:33.861]
[22:47:33.861]
[22:47:33.861] > Build failed because of webpack errors
[22:47:33.885] Error: Command "npm run build" exited with 1

@SaptarshiSarkar12 can you please show me the latest logs? I'm not able to sign into vercel

Here's the latest logs.
[22:49:23.768]    Creating an optimized production build ...
[22:49:26.047] Failed to compile.
[22:49:26.047]
[22:49:26.047] ./app/docs/page.js
[22:49:26.047] ReactServerComponentsError:
[22:49:26.047]
[22:49:26.047] "getStaticProps" is not supported in app/. Read more: https://nextjs.org/docs/app/building-your-application/data-fetching
[22:49:26.047]
[22:49:26.048]     ,-[/vercel/path0/Website/app/docs/page.js:23:1]
[22:49:26.048]  23 |   )
[22:49:26.048]  24 | }
[22:49:26.048]  25 |
[22:49:26.048]  26 | export async function getStaticProps() {
[22:49:26.048]     :                       ^^^^^^^^^^^^^^
[22:49:26.048]  27 |   const headerHtml = await getComponentHtml('header')
[22:49:26.048]  28 |   const quickstartHtml = await getComponentHtml('quickstart')
[22:49:26.048]  29 |   const gettingStartedHtml = await getComponentHtml('getting-started')
[22:49:26.048]     `----
[22:49:26.048]
[22:49:26.048] File path:
[22:49:26.048]   ./app/docs/page.js
[22:49:26.048]
[22:49:26.049]
[22:49:26.049] > Build failed because of webpack errors
[22:49:26.072] Error: Command "npm run build" exited with 1

@markandey1414 I have disabled Log protection. So, you can check the logs and find what is the reason for the error and applying the required fix.

You do not require to login now.

@markandey1414 Try running npm i react-markdown in the Website directory and then, commit the changes. The error will get resolved.

@SaptarshiSarkar12 I'll be going further with this issue, will resolve it 👍

@markandey1414 Try running npm i react-markdown in the Website directory and then, commit the changes. The error will get resolved.

its there already, I've just added it in gitignore

Okay @markandey1414

@markandey1414 Try running npm i react-markdown in the Website directory and then, commit the changes. The error will get resolved.

its there already, I've just added it in gitignore

@markandey1414 Please remove any changes that you have made in .gitignore file. Node Modules are already added in gitignore file present in the Wesbite directory.

@markandey1414 Were you able to understand the reason for the build failure? I think you have not added react-markdown in your package.json file. Also, the lib/api.js file no longer exists. So, you need to fix these issues to have a successful deployment.

@markandey1414 Were you able to understand the reason for the build failure? I think you have not added react-markdown in your package.json file. Also, the lib/api.js file no longer exists. So, you need to fix these issues to have a successful deployment.

yes I did understand the issues with docs/page.js. But the file api.js is still there, I guess there's no issue with that.

@SaptarshiSarkar12 I'm facing an issue while pushing changes. I even added node modules to gitignore and did all I could think, yet I'm repeatedly getting this issue.

@markandey1414 Please don't push. I think you are trying to push the node_modules directory which is not good. So, let me fix the error. Then, everything should work fine on your end also. I am trying to fix the lib and gitignore files.

yes sure, I tried the pull command yet no changes.

Ok. Thank you

@markandey1414 You can now start working on the docs page. Some part of the api.js file is not correct, for which the build is failing. Please resolve that issue. Also, the docs page will not be single page. It will have multiple sections and sub-sections, so, please try to make it accordingly.

@markandey1414 Did you face any problem after I have pushed my changes?

@SaptarshiSarkar12 I tried multiple times, but it has been the same issue over and over again

@markandey1414 You can see this tutorial on how to use MDX for next js website and try re-creating the docs section again. Please use MDX as said in that video. It works.

@markandey1414 Are you able to solve this issue? Have you seen the resource I have shared earlier?

@markandey1414 Please resolve the branch conflicts

@markandey1414 Please respond.

@markandey1414 Do you still wish to work on it? I shall close this PR as you have been inactive for a long time.

Hello @SaptarshiSarkar12, apologies for late response and all the trouble. I would like to withdraw this pr as I'm not able to solve the issue for a long time.

It's okay. Thank you for the response 🙂. I am closing this PR.


Issue on the column lineage from SQL Server views

SQL Server metadata ingestion module
The impact is on ingestion of metadata from SQL Server views and the resulting lineage
Describe the bug
After some tests with the help of @harshach , it has been understood that the SQL parser used in the metadata ingestion from a SQL Server view definition does not handles correctly the square brackets surrounding the column names.  The issue is reflected on the lineage which does not displays any of the column "level" lineage flows for such columns having square brackets on their names:

Bellow, you'll find an example of the SQL code for a view in SQL Server using the "square brackets" syntax, which causes issues to the parser, as well as a syntax without issues for the parser.