Mixtral AWQ fails to work: asyncio.exceptions.CancelledError: Cancelled by cancel scope 7fd214489990

export CUDA_HOME=/usr/local/cuda-12.3
export PIP_EXTRA_INDEX_URL="https://download.pytorch.org/whl/cu123"
pip install git+https://github.com/vllm-project/vllm.git --upgrade
export CUDA_VISIBLE_DEVICES=1

python -m vllm.entrypoints.openai.api_server --port=5002 --host=<IP_ADDRESS> --model TheBloke/Mixtral-8x7B-Instruct-v0.1-AWQ --quantization awq --dtype auto --seed 1234 --tensor-parallel-size=1 --max-num-batched-tokens=66560 --max-log-len=100

Any where, even simple, leads to:
INFO 01-27 01:15:31 api_server.py:209] args: Namespace(host='<IP_ADDRESS>', port=5002, allow_credentials=False, allowed_origins=['*'], allowed_methods=['*'], allowed_headers=['*'], api_key=None, served_model_name=None, chat_template=None, response_role='assistant', ssl_keyfile=None, ssl_certfile=None, root_path=None, middleware=[], model='TheBloke/Mixtral-8x7B-Instru>
WARNING 01-27 01:15:31 config.py:176] awq quantization is not fully optimized yet. The speed can be slower than non-quantized models.
INFO 01-27 01:15:31 llm_engine.py:72] Initializing an LLM engine with config: model='TheBloke/Mixtral-8x7B-Instruct-v0.1-AWQ', tokenizer='TheBloke/Mixtral-8x7B-Instruct-v0.1-AWQ', tokenizer_mode=auto, revision=None, tokenizer_revision=None, trust_remote_code=False, dtype=torch.float16, max_seq_len=32768, download_dir=None, load_format=auto, tensor_parallel_size=>
INFO 01-27 01:15:33 weight_utils.py:164] Using model weights format ['*.safetensors']
INFO 01-27 01:17:50 llm_engine.py:316] # GPU blocks: 12486, # CPU blocks: 2048
INFO 01-27 01:17:51 model_runner.py:625] Capturing the model for CUDA graphs. This may lead to unexpected consequences if the model is not static. To run the model in eager mode, set 'enforce_eager=True' or use '--enforce-eager' in the CLI.
INFO 01-27 01:17:51 model_runner.py:629] CUDA graphs can take additional 1~3 GiB memory per GPU. If you are running out of memory, consider decreasing `gpu_memory_utilization` or enforcing eager mode. You can also reduce the `max_num_seqs` as needed to decrease memory usage.
INFO 01-27 01:18:04 model_runner.py:689] Graph capturing finished in 13 secs.
INFO 01-27 01:18:04 serving_chat.py:260] Using default chat template:
INFO 01-27 01:18:04 serving_chat.py:260] {{ bos_token }}{% for message in messages %}{% if (message['role'] == 'user') != (loop.index0 % 2 == 0) %}{{ raise_exception('Conversation roles must alternate user/assistant/user/assistant/...') }}{% endif %}{% if message['role'] == 'user' %}{{ '[INST] ' + message['content'] + ' [/INST]' }}{% elif message['role'] == 'ass>
INFO:     Started server process [276444]
INFO:     Waiting for application startup.
INFO:     Application startup complete.
INFO:     Uvicorn running on http://<IP_ADDRESS>:5002 (Press CTRL+C to quit)
INFO 01-27 01:18:14 llm_engine.py:871] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%
INFO 01-27 01:18:24 llm_engine.py:871] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%
INFO 01-27 01:18:34 llm_engine.py:871] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%
INFO 01-27 01:18:41 async_llm_engine.py:433] Received request cmpl-cd9d75c607614e7db704b01164bc0c83-0: prompt: None, prefix_pos: None,sampling params: SamplingParams(n=1, best_of=1, presence_penalty=0.14000000000000012, frequency_penalty=0.0, repetition_penalty=1.0, temperature=0.0, top_p=1.0, top_k=-1, min_p=0.0, use_beam_search=False, length_penalty=1.0, early>
INFO:     <IP_ADDRESS>:43684 - "POST /v1/completions HTTP/1.1" 200 OK
INFO 01-27 01:18:41 llm_engine.py:871] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%
ERROR:    Exception in ASGI application
Traceback (most recent call last):
  File "/ephemeral/vllm/lib/python3.10/site-packages/starlette/responses.py", line 261, in __call__
    await wrap(partial(self.listen_for_disconnect, receive))
  File "/ephemeral/vllm/lib/python3.10/site-packages/starlette/responses.py", line 257, in wrap
    await func()
  File "/ephemeral/vllm/lib/python3.10/site-packages/starlette/responses.py", line 234, in listen_for_disconnect
    message = await receive()
  File "/ephemeral/vllm/lib/python3.10/site-packages/uvicorn/protocols/http/httptools_impl.py", line 580, in receive
    await self.message_event.wait()
  File "/ephemeral/vllm/lib/python3.10/asyncio/locks.py", line 214, in wait
    await fut
asyncio.exceptions.CancelledError: Cancelled by cancel scope 7fd214489990

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/ephemeral/vllm/lib/python3.10/site-packages/uvicorn/protocols/http/httptools_impl.py", line 419, in run_asgi
    result = await app(  # type: ignore[func-returns-value]
  File "/ephemeral/vllm/lib/python3.10/site-packages/uvicorn/middleware/proxy_headers.py", line 84, in __call__
    return await self.app(scope, receive, send)
  File "/ephemeral/vllm/lib/python3.10/site-packages/fastapi/applications.py", line 1054, in __call__
    await super().__call__(scope, receive, send)
  File "/ephemeral/vllm/lib/python3.10/site-packages/starlette/applications.py", line 123, in __call__
    await self.middleware_stack(scope, receive, send)
  File "/ephemeral/vllm/lib/python3.10/site-packages/starlette/middleware/errors.py", line 186, in __call__
    raise exc
  File "/ephemeral/vllm/lib/python3.10/site-packages/starlette/middleware/errors.py", line 164, in __call__
    await self.app(scope, receive, _send)
  File "/ephemeral/vllm/lib/python3.10/site-packages/starlette/middleware/cors.py", line 83, in __call__
    await self.app(scope, receive, send)
  File "/ephemeral/vllm/lib/python3.10/site-packages/aioprometheus/asgi/middleware.py", line 184, in __call__
    await self.asgi_callable(scope, receive, wrapped_send)
  File "/ephemeral/vllm/lib/python3.10/site-packages/starlette/middleware/exceptions.py", line 62, in __call__
    await wrap_app_handling_exceptions(self.app, conn)(scope, receive, send)
  File "/ephemeral/vllm/lib/python3.10/site-packages/starlette/_exception_handler.py", line 64, in wrapped_app
    raise exc
  File "/ephemeral/vllm/lib/python3.10/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
    await app(scope, receive, sender)
  File "/ephemeral/vllm/lib/python3.10/site-packages/starlette/routing.py", line 762, in __call__
    await self.middleware_stack(scope, receive, send)
  File "/ephemeral/vllm/lib/python3.10/site-packages/starlette/routing.py", line 782, in app
    await route.handle(scope, receive, send)
  File "/ephemeral/vllm/lib/python3.10/site-packages/starlette/routing.py", line 297, in handle
    await self.app(scope, receive, send)
  File "/ephemeral/vllm/lib/python3.10/site-packages/starlette/routing.py", line 77, in app
    await wrap_app_handling_exceptions(app, request)(scope, receive, send)
  File "/ephemeral/vllm/lib/python3.10/site-packages/starlette/_exception_handler.py", line 64, in wrapped_app
    raise exc
  File "/ephemeral/vllm/lib/python3.10/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
    await app(scope, receive, sender)
  File "/ephemeral/vllm/lib/python3.10/site-packages/starlette/routing.py", line 75, in app
    await response(scope, receive, send)
  File "/ephemeral/vllm/lib/python3.10/site-packages/starlette/responses.py", line 254, in __call__
    async with anyio.create_task_group() as task_group:
  File "/ephemeral/vllm/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 678, in __aexit__
    raise BaseExceptionGroup(
exceptiongroup.ExceptionGroup: unhandled errors in a TaskGroup (1 sub-exception)
INFO 01-27 01:18:46 llm_engine.py:871] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 19.6 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.1%, CPU KV cache usage: 0.0%
INFO 01-27 01:18:51 llm_engine.py:871] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 19.5 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.1%, CPU KV cache usage: 0.0%
INFO 01-27 01:18:56 llm_engine.py:871] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 19.5 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.2%, CPU KV cache usage: 0.0%
INFO 01-27 01:19:01 llm_engine.py:871] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 19.4 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.2%, CPU KV cache usage: 0.0%
INFO 01-27 01:19:06 llm_engine.py:871] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 19.4 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.3%, CPU KV cache usage: 0.0%
INFO 01-27 01:19:11 llm_engine.py:871] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 19.4 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.3%, CPU KV cache usage: 0.0%
INFO 01-27 01:19:16 llm_engine.py:871] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 19.4 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.4%, CPU KV cache usage: 0.0%
INFO 01-27 01:19:21 llm_engine.py:871] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 19.4 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.4%, CPU KV cache usage: 0.0%
INFO 01-27 01:19:26 llm_engine.py:871] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 19.4 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.5%, CPU KV cache usage: 0.0%
INFO 01-27 01:19:31 llm_engine.py:871] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 19.4 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.5%, CPU KV cache usage: 0.0%
INFO 01-27 01:19:33 async_llm_engine.py:112] Finished request cmpl-cd9d75c607614e7db704b01164bc0c83-0.
INFO 01-27 01:19:44 llm_engine.py:871] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%
INFO 01-27 01:19:54 llm_engine.py:871] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%
INFO 01-27 01:20:04 llm_engine.py:871] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%
INFO 01-27 01:20:14 llm_engine.py:871] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%
INFO 01-27 01:20:24 llm_engine.py:871] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%

When I try to use a different model, I get other errors:
export CUDA_HOME=/usr/local/cuda-12.3
export PIP_EXTRA_INDEX_URL="https://download.pytorch.org/whl/cu123"
pip install git+https://github.com/vllm-project/vllm.git --upgrade
export CUDA_VISIBLE_DEVICES=1

python -m vllm.entrypoints.openai.api_server --port=5002 --host=<IP_ADDRESS> --model casperhansen/mixtral-instruct-awq --quantization awq --dtype auto --seed 1234 --tensor-parallel-size=1 --max-num-batched-tokens=66560 --max-log-len=100

INFO:     <IP_ADDRESS>:53634 - "POST /v1/completions HTTP/1.1" 500 Internal Server Error
ERROR:    Exception in ASGI application
Traceback (most recent call last):
  File "/ephemeral/vllm/lib/python3.10/site-packages/uvicorn/protocols/http/httptools_impl.py", line 419, in run_asgi
    result = await app(  # type: ignore[func-returns-value]
  File "/ephemeral/vllm/lib/python3.10/site-packages/uvicorn/middleware/proxy_headers.py", line 84, in __call__
    return await self.app(scope, receive, send)
  File "/ephemeral/vllm/lib/python3.10/site-packages/fastapi/applications.py", line 1054, in __call__
    await super().__call__(scope, receive, send)
  File "/ephemeral/vllm/lib/python3.10/site-packages/starlette/applications.py", line 123, in __call__
    await self.middleware_stack(scope, receive, send)
  File "/ephemeral/vllm/lib/python3.10/site-packages/starlette/middleware/errors.py", line 186, in __call__
    raise exc
  File "/ephemeral/vllm/lib/python3.10/site-packages/starlette/middleware/errors.py", line 164, in __call__
    await self.app(scope, receive, _send)
  File "/ephemeral/vllm/lib/python3.10/site-packages/starlette/middleware/cors.py", line 83, in __call__
    await self.app(scope, receive, send)
  File "/ephemeral/vllm/lib/python3.10/site-packages/aioprometheus/asgi/middleware.py", line 184, in __call__
    await self.asgi_callable(scope, receive, wrapped_send)
  File "/ephemeral/vllm/lib/python3.10/site-packages/starlette/middleware/exceptions.py", line 62, in __call__
    await wrap_app_handling_exceptions(self.app, conn)(scope, receive, send)
  File "/ephemeral/vllm/lib/python3.10/site-packages/starlette/_exception_handler.py", line 64, in wrapped_app
    raise exc
  File "/ephemeral/vllm/lib/python3.10/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
    await app(scope, receive, sender)
  File "/ephemeral/vllm/lib/python3.10/site-packages/starlette/routing.py", line 762, in __call__
    await self.middleware_stack(scope, receive, send)
  File "/ephemeral/vllm/lib/python3.10/site-packages/starlette/routing.py", line 782, in app
    await route.handle(scope, receive, send)
  File "/ephemeral/vllm/lib/python3.10/site-packages/starlette/routing.py", line 297, in handle
    await self.app(scope, receive, send)
  File "/ephemeral/vllm/lib/python3.10/site-packages/starlette/routing.py", line 77, in app
    await wrap_app_handling_exceptions(app, request)(scope, receive, send)
  File "/ephemeral/vllm/lib/python3.10/site-packages/starlette/_exception_handler.py", line 64, in wrapped_app
    raise exc
  File "/ephemeral/vllm/lib/python3.10/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
    await app(scope, receive, sender)
  File "/ephemeral/vllm/lib/python3.10/site-packages/starlette/routing.py", line 72, in app
    response = await func(request)
  File "/ephemeral/vllm/lib/python3.10/site-packages/fastapi/routing.py", line 299, in app
    raise e
  File "/ephemeral/vllm/lib/python3.10/site-packages/fastapi/routing.py", line 294, in app
    raw_response = await run_endpoint_function(
  File "/ephemeral/vllm/lib/python3.10/site-packages/fastapi/routing.py", line 191, in run_endpoint_function
    return await dependant.call(**values)
  File "/ephemeral/vllm/lib/python3.10/site-packages/vllm/entrypoints/openai/api_server.py", line 166, in create_completion
    return JSONResponse(content=generator.model_dump(),
AttributeError: 'ErrorResponse' object has no attribute 'model_dump'

Same exception for https://huggingface.co/TheBloke/Nous-Hermes-2-Mixtral-8x7B-DPO-AWQ
INFO 01-27 01:46:22 llm_engine.py:316] # GPU blocks: 18691, # CPU blocks: 2048
INFO 01-27 01:46:23 model_runner.py:625] Capturing the model for CUDA graphs. This may lead to unexpected consequences if the model is not static. To run the model in eager mode, set 'enforce_eager=True' or use '--enforce-eager' in the CLI.
INFO 01-27 01:46:23 model_runner.py:629] CUDA graphs can take additional 1~3 GiB memory per GPU. If you are running out of memory, consider decreasing `gpu_memory_utilization` or enforcing eager mode. You can also reduce the `max_num_seqs` as needed to decreas>
INFO 01-27 01:46:36 model_runner.py:689] Graph capturing finished in 13 secs.
Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.
INFO 01-27 01:46:36 serving_chat.py:260] Using default chat template:
INFO 01-27 01:46:36 serving_chat.py:260] {% for message in messages %}{{'<|im_start|>' + message['role'] + '
INFO 01-27 01:46:36 serving_chat.py:260] ' + message['content'] + '<|im_end|>' + '
INFO 01-27 01:46:36 serving_chat.py:260] '}}{% endfor %}{% if add_generation_prompt %}{{ '<|im_start|>assistant
INFO 01-27 01:46:36 serving_chat.py:260] ' }}{% endif %}
Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.
INFO:     Started server process [277174]
INFO:     Waiting for application startup.
INFO:     Application startup complete.
INFO:     Uvicorn running on http://<IP_ADDRESS>:5002 (Press CTRL+C to quit)
INFO 01-27 01:46:46 llm_engine.py:871] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%
INFO 01-27 01:46:56 llm_engine.py:871] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%
INFO 01-27 01:47:06 llm_engine.py:871] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%
INFO 01-27 01:47:16 llm_engine.py:871] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%
INFO 01-27 01:47:26 llm_engine.py:871] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%
INFO 01-27 01:47:36 llm_engine.py:871] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%
INFO 01-27 01:47:46 llm_engine.py:871] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%
INFO 01-27 01:47:56 llm_engine.py:871] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%
INFO 01-27 01:48:06 llm_engine.py:871] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%
INFO 01-27 01:48:16 llm_engine.py:871] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%
INFO 01-27 01:48:26 llm_engine.py:871] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%
INFO 01-27 01:48:36 llm_engine.py:871] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%
INFO 01-27 01:48:46 llm_engine.py:871] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%
INFO 01-27 01:48:56 llm_engine.py:871] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%
INFO 01-27 01:49:06 llm_engine.py:871] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%
INFO 01-27 01:49:16 llm_engine.py:871] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%
INFO 01-27 01:49:26 llm_engine.py:871] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%
INFO 01-27 01:49:36 llm_engine.py:871] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%
INFO 01-27 01:49:46 llm_engine.py:871] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%
INFO 01-27 01:49:56 llm_engine.py:871] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%
INFO 01-27 01:50:06 llm_engine.py:871] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%
INFO 01-27 01:50:16 llm_engine.py:871] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%
INFO 01-27 01:50:26 llm_engine.py:871] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%
INFO 01-27 01:50:36 llm_engine.py:871] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%
INFO 01-27 01:50:46 llm_engine.py:871] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%
INFO 01-27 01:50:56 llm_engine.py:871] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%
INFO 01-27 01:51:06 llm_engine.py:871] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%
INFO 01-27 01:51:16 llm_engine.py:871] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%
INFO 01-27 01:51:22 async_llm_engine.py:433] Received request cmpl-f9ebefcdb0b940ff9edb641a0cc03634-0: prompt: None, prefix_pos: None,sampling params: SamplingParams(n=1, best_of=1, presence_penalty=0.14000000000000012, frequency_penalty=0.0, repetition_penalt>
INFO:     <IP_ADDRESS>:50012 - "POST /v1/completions HTTP/1.1" 200 OK
INFO 01-27 01:51:22 llm_engine.py:871] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%
ERROR:    Exception in ASGI application
Traceback (most recent call last):
  File "/ephemeral/vllm/lib/python3.10/site-packages/starlette/responses.py", line 261, in __call__
    await wrap(partial(self.listen_for_disconnect, receive))
  File "/ephemeral/vllm/lib/python3.10/site-packages/starlette/responses.py", line 257, in wrap
    await func()
  File "/ephemeral/vllm/lib/python3.10/site-packages/starlette/responses.py", line 234, in listen_for_disconnect
    message = await receive()
  File "/ephemeral/vllm/lib/python3.10/site-packages/uvicorn/protocols/http/httptools_impl.py", line 580, in receive
    await self.message_event.wait()
  File "/ephemeral/vllm/lib/python3.10/asyncio/locks.py", line 214, in wait
    await fut
asyncio.exceptions.CancelledError: Cancelled by cancel scope 7f318c3b1900

During handling of the above exception, another exception occurred:

Traceback (most recent call last):

On 0.2.7 release, never get any answer back even if seems to generate.

TheBloke/Mixtral-8x7B-Instruct-v0.1-AWQ on 0.2.7 just hangs in generation. never shows any output.

When I try to use a different model, I get other errors:
export CUDA_HOME=/usr/local/cuda-12.3
export PIP_EXTRA_INDEX_URL="https://download.pytorch.org/whl/cu123"
pip install git+https://github.com/vllm-project/vllm.git --upgrade
export CUDA_VISIBLE_DEVICES=1

python -m vllm.entrypoints.openai.api_server --port=5002 --host=<IP_ADDRESS> --model casperhansen/mixtral-instruct-awq --quantization awq --dtype auto --seed 1234 --tensor-parallel-size=1 --max-num-batched-tokens=66560 --max-log-len=100

INFO:     <IP_ADDRESS>:53634 - "POST /v1/completions HTTP/1.1" 500 Internal Server Error
ERROR:    Exception in ASGI application
Traceback (most recent call last):
  File "/ephemeral/vllm/lib/python3.10/site-packages/uvicorn/protocols/http/httptools_impl.py", line 419, in run_asgi
    result = await app(  # type: ignore[func-returns-value]
  File "/ephemeral/vllm/lib/python3.10/site-packages/uvicorn/middleware/proxy_headers.py", line 84, in __call__
    return await self.app(scope, receive, send)
  File "/ephemeral/vllm/lib/python3.10/site-packages/fastapi/applications.py", line 1054, in __call__
    await super().__call__(scope, receive, send)
  File "/ephemeral/vllm/lib/python3.10/site-packages/starlette/applications.py", line 123, in __call__
    await self.middleware_stack(scope, receive, send)
  File "/ephemeral/vllm/lib/python3.10/site-packages/starlette/middleware/errors.py", line 186, in __call__
    raise exc
  File "/ephemeral/vllm/lib/python3.10/site-packages/starlette/middleware/errors.py", line 164, in __call__
    await self.app(scope, receive, _send)
  File "/ephemeral/vllm/lib/python3.10/site-packages/starlette/middleware/cors.py", line 83, in __call__
    await self.app(scope, receive, send)
  File "/ephemeral/vllm/lib/python3.10/site-packages/aioprometheus/asgi/middleware.py", line 184, in __call__
    await self.asgi_callable(scope, receive, wrapped_send)
  File "/ephemeral/vllm/lib/python3.10/site-packages/starlette/middleware/exceptions.py", line 62, in __call__
    await wrap_app_handling_exceptions(self.app, conn)(scope, receive, send)
  File "/ephemeral/vllm/lib/python3.10/site-packages/starlette/_exception_handler.py", line 64, in wrapped_app
    raise exc
  File "/ephemeral/vllm/lib/python3.10/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
    await app(scope, receive, sender)
  File "/ephemeral/vllm/lib/python3.10/site-packages/starlette/routing.py", line 762, in __call__
    await self.middleware_stack(scope, receive, send)
  File "/ephemeral/vllm/lib/python3.10/site-packages/starlette/routing.py", line 782, in app
    await route.handle(scope, receive, send)
  File "/ephemeral/vllm/lib/python3.10/site-packages/starlette/routing.py", line 297, in handle
    await self.app(scope, receive, send)
  File "/ephemeral/vllm/lib/python3.10/site-packages/starlette/routing.py", line 77, in app
    await wrap_app_handling_exceptions(app, request)(scope, receive, send)
  File "/ephemeral/vllm/lib/python3.10/site-packages/starlette/_exception_handler.py", line 64, in wrapped_app
    raise exc
  File "/ephemeral/vllm/lib/python3.10/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
    await app(scope, receive, sender)
  File "/ephemeral/vllm/lib/python3.10/site-packages/starlette/routing.py", line 72, in app
    response = await func(request)
  File "/ephemeral/vllm/lib/python3.10/site-packages/fastapi/routing.py", line 299, in app
    raise e
  File "/ephemeral/vllm/lib/python3.10/site-packages/fastapi/routing.py", line 294, in app
    raw_response = await run_endpoint_function(
  File "/ephemeral/vllm/lib/python3.10/site-packages/fastapi/routing.py", line 191, in run_endpoint_function
    return await dependant.call(**values)
  File "/ephemeral/vllm/lib/python3.10/site-packages/vllm/entrypoints/openai/api_server.py", line 166, in create_completion
    return JSONResponse(content=generator.model_dump(),
AttributeError: 'ErrorResponse' object has no attribute 'model_dump'

Encountered the same problem，i use Qwen/Qwen-7B-Chat model。It seems that vllm.entrypoints.openai.api_server.py  is not compatible。
When the service is first started, there is also an error：AttributeError: 'AsyncLLMEngine' object has no attribute 'do_log_stats'，I have already used the latest  0.2.7 release

The error
AttributeError: 'ErrorResponse' object has no attribute 'model_dump'

seems related to the version of pydantic which was recently upgraded #2531

@pseudotensor When using Qwen1.5 awq model, I still have this problem.

doesn't seem to be model nor quantization specific, got the same error now using TechxGenus/starcoder2-7b-GPTQ.
I'm using the latest vllm docker image with the following arguments:
--model "TechxGenus/starcoder2-7b-GPTQ" --revision 48dc06e6a6df8a8e8567694ead23f59204fa0d26 -q marlin --enable-chunked-prefill --max-model-len 4096 --gpu-memory-utilization 0.3
last week it still worked, so not sure what the reason is.
tgi-scripts-vllm-starcoder-2-7b-gptq-1  | INFO 05-28 09:47:25 llm_engine.py:100] Initializing an LLM engine (v0.4.2) with config: model='TechxGenus/starcoder2-7b-GPTQ', speculative_config=None, tokenizer='TechxGenus/starcoder2-7b-GPTQ', skip_tokenizer_init=False, tokenizer_mode=auto, revision=48dc06e6a6df8a8e8567694ead23f59204fa0d26, tokenizer_revision=None, trust_remote_code=False, dtype=torch.float16, max_seq_len=4096, download_dir=None, load_format=LoadFormat.AUTO, tensor_parallel_size=1, disable_custom_all_reduce=False, quantization=gptq_marlin, enforce_eager=False, kv_cache_dtype=auto, quantization_param_path=None, device_config=cuda, decoding_config=DecodingConfig(guided_decoding_backend='outlines'), seed=0, served_model_name=TechxGenus/starcoder2-7b-GPTQ)

FWIW, I had this CancelledError: Cancelled by cancel scope error, and I solved it by changing --gpu-memory-utilization 0.99 to --gpu-memory-utilization 0.96, so it might have to do with some sort of silent allocation failure that is helped by having a bit of a "buffer". In my case I had two 4090s on Runpod using vllm/vllm-openai:v0.4.3, running a 70B GPTQ model. These were the other command line options that I had:
--quantization gptq --dtype float16 --enforce-eager --tensor-parallel-size 2 --max-model-len 4096 --kv-cache-dtype fp8

@hmellor @pseudotensor Maybe this could this be re-opened for the benefit of others, since two others have recently commented with similar issues since it was closed.
Potentially related:

https://github.com/vllm-project/vllm/issues/5182

The same generic error in starlette does not mean the problem is the same, especially when the original error was reported 4 months ago using a (now) old version of vLLM.
The different traces shared in the thread comments are not that similar.
If someone is experiencing the error they should open an issue with instructions on how to reproduce the error.


Using MapKey in addition with property

I am currently working on a business project with extremely little control over the database structure. The database has the following tables: Item, VendorItemItem. Item is a simple entity table and VendorItemItem is a table linking Item and VendorItem together.
These two tables link to eachother, but not by traditional means. The foreign key in VendorItemItem does not link to the primary key of Item. Instead it links to another foreign key in Item.
Item
Id
OtherId
VendorItemItem
ItemOtherId
VendorItemId
This means I would like to use MapKey to accomplish this relationship with code first. However, MapKey apparently requires that the property is not used. Is there a way around this so I can still have the foreign key as a property in my Item object? Other applications will depend on this property and will not be able to insert records into VendorItemItem if it is not available.
Thank you for your time!

@bdebaere This is a limitation in EF6 that is, unfortunately, not ever likely to get removed. Sometimes it can be worked around by changing which property EF thinks is the PK--for example try mapping OtherId as the PK.
EF Core supports this kind of relationship, but may not be suitable for other reasons.

@ajcvickers Thank you for your response. I cannot use this OtherId as PK because other tables are linked with the Id unfortunately.
MapToStoredProcedures and Entity splitting (hopefully with optional relationships) is not yet supported in EF Core as far as I know so we cannot switch yet.
If there is no possible workaround, then I thank you for your time.


Anchor Tag Helpers in navbar-link?

There are any plan to implements built-in anchor tag helpers in <navbar-link>
Example for razor pages:
<navbar-link asp-page="/">Index</navbar-link>
Example for MVC:
<navbar-link asp-controller="Home" asp-action="Index">Index</navbar-link>
Regards

We are happy to include this in our Release Planning. Thank you for feedback.

Thank you!


Make cli flag an actual flag

In the current version of the makepackage package, to create a package with the command-line interface, you need to add cli argument to the makepackage command, like this:
$ makepackage newpackage cli

This does not look as a typical flag used in Python for command-line interface, does it? So, I propose to make it
$ makepackage newpackage --cli

so the flag is what we consider a command-line flag indeed. To keep backwards compatibility, the makepackage command should also accept the cli flag, and, because why not, the -cli flag.
To summarize, three versions of the flag will be accepted:

cli
-cli
--cli

Note that they cannot be confused with anything else, so introducing this change will not be dangerous in any way. I think we can keep the three flags; perhaps in the future we can remove the cli flag, though I really do not see the point, so I am reluctant to the idea.
@hp0404 and @AlixBernard , I'd be obliged for your thoughts on the issue.

If it can't break anything and is convenient, why not? I think all three should be accepted

Done.

I believe this issue is still open in the main branch, which pip install will get from pypi.
As-is, the usage is makepackage <name> ["cli"] only, --cli isn't recognized. There's an improve-cli-flag branch, which I assume fixes this, but it needs merging into main to install from pypi.

Yes, indeed, thanks for spotting this! I will take care of this soon.

After all, it's truly done and closed.


Faker instance func type

Alternate option for #130.

https://github.com/brianvoe/gofakeit/pull/134
We can pick up from here.

https://github.com/brianvoe/gofakeit/pull/134
We can pick up from here.


Update the kong pongo git url

Please update the kong-pongo git clone url to
git clone https://github.com/Kong/kong-plugin.git ,
from git clone git@github.com:Kong/kong-pongo.git
all users might not able to clone from it .

what is the problem?

what is the problem?

Permission denied when cloning from the git@github.com:Kong/kong-pongo.git

Permission denied when cloning from the git@github.com:Kong/kong-pongo.git

shouldn't that be identical? I've never seen that. Any links to docs?
also what references should be updated?

shouldn't that be identical? I've never seen that. Any links to docs?
also what references should be updated?

@Tieske  The document mention the git clone url as
git clone git@github.com:Kong/kong-pongo.git
when users try to clone it , they get permission denied as mentioned in the attachment .
Instead of this we can mention http/s url to clone so that no user will face issue related to permission

@Tieske  The document mention the git clone url as
git clone git@github.com:Kong/kong-pongo.git
when users try to clone it , they get permission denied as mentioned in the attachment .
Instead of this we can mention http/s url to clone so that no user will face issue related to permission

@Tieske  adding more details these git clone URL , require update
https://github.com/Kong/kong-pongo#installation
change git clone git@github.com:Kong/kong-pongo.git to
https://github.com/Kong/kong-pongo.git
as step to add ssh key is missing  or we have to add step to add ssh key from github
same update goes for : https://github.com/Kong/kong-pongo#do-a-test-run
where git clone git@github.com:Kong/kong-plugin.git  require change .
same updates goes for : https://github.com/Kong/kong-pongo#pongo-on-windows
where git clone git@github.com:Kong/kong-pongo.git require update
In this section we are using correct git clone URL
https://github.com/Kong/kong-pongo#a-walkthrough

@Tieske  adding more details these git clone URL , require update
https://github.com/Kong/kong-pongo#installation
change git clone git@github.com:Kong/kong-pongo.git to
https://github.com/Kong/kong-pongo.git
as step to add ssh key is missing  or we have to add step to add ssh key from github
same update goes for : https://github.com/Kong/kong-pongo#do-a-test-run
where git clone git@github.com:Kong/kong-plugin.git  require change .
same updates goes for : https://github.com/Kong/kong-pongo#pongo-on-windows
where git clone git@github.com:Kong/kong-pongo.git require update
In this section we are using correct git clone URL
https://github.com/Kong/kong-pongo#a-walkthrough

ok thx, would you mind sending a PR?

ok thx, would you mind sending a PR?

Hi @Tieske  i added new PR for the changes
https://github.com/Kong/kong-pongo/pull/144

Hi @Tieske  i added new PR for the changes
https://github.com/Kong/kong-pongo/pull/144


Style enhancements for header font and spacing between list items

Here are a couple of enhancements that have been an issue while using MarkdownUI.

Add a feature to provide more control over the font used for headers
Add a feature to control the space between list items

Hi @jefferythomas and @emartinson,
Please take a look at the latest release:
https://github.com/gonzalezreal/swift-markdown-ui/releases/tag/2.0.0
This version is a game-changer in terms of styling. You can do what you requested and much more.


[Feature] Add TensorRT benchmark

Hi,
First of all thanks a lot for this amazing project.
At some point, it would be interesting to add pure TensorRT benchmark (without passing by ORT) as FP32, FP16, FP8 and INT8.
Let me know if you might find this interesting. I will be happy to help.

is there a direct way to use the transformers api on tensorrt ?

Not as far as I know. What I'm basically doing is to turn a PT model into ONNX with optimum and then turn this ONNX model into TRT.

For now we're focusing on backends and devices supported through optimum.
With just the ONNX/TRT file you can't perform tasks like text-generation, automatic-speech-recognition, etc. It would only work for feature-extraction at best.
@regisss what do you think ?

Yes, I agree with @IlyasMoutawwakil that for now we should restrict ourselves to backends and devices supported by Optimum.

@IlyasMoutawwakil I agree that static graph compilation methods will not be easily automated. However, as of Torch-TensorRT v1.4.0 there is support for the torch.compile workflow in TensorRT which would allow us to invoke the same feature set supported in the Pytorch Eager / compiled Backend: https://github.com/huggingface/optimum-benchmark/blob/main/optimum_benchmark/backends/pytorch.py#L123
The call is similar, yet we use the torch_tensorrt package for compilation instead of native Pytorch:
import torch_tensorrt
torch_tensorrt.dynamo.compile(model, inputs, ...)

##### OR #####

torch_tensorrt.compile(model, ir="dynamo_compile", inputs=inputs, ...)

Would a solution like this be acceptable? @jplu @regisss

@x11kjm Interesting! Would you like to open a PR to add this to Optimum?
@fxmarty WDYT?

I would like to work on this if it's still needed as a feature


Status page Delete incident confirmation modal appears at bottom of page

The confirmation modal window appears more as a white box at the bottom of the page rather than a modal over the top of other elements. Either this should take you to the dashboard and trigger the delete action or the confirmation should appear correctly as a modal.
Tested in Chrome and FF.

CSS added back in. Now works as expected in Chrome and FF.


cs hud closes immediately

c:\cs2hud>cs-hud.exe
c:\cs2hud>
(node:7860) electron: Failed to load URL: http://localhost:31982/hud?transparent&corners with error: ERR_CONNECTION_REFUSED
(Use `cs-hud --trace-warnings ...` to show where the warning was created)
[7860:0213/231303.669:ERROR:interface_endpoint_client.cc(702)] Message 0 rejected by interface blink.mojom.WidgetHost

when start with cmd

You also need to download cs-hud-win.exe from the Release page, and run it first.


Helper Commands

The Kakapo bot should have a group of easy-to-make helper commands. For example, if one types !ip, the bot should then display the IP address of the server, the port of our Arma 3 servers, and how to connect to them all within an embed message.
The following image gives one example of a helper message. Notice the concise language used and how it fit within an embed message.

The following commands should be added. It is up to you to decide the formatting and message content. All that matters is it gets the point across. It is also important that other bot developers can easily add new helper commands.
NOTE: If you see multiple, similar commands, they are aliases of the same function. For example !mod and !mods display the same message.
!ip
!ipaddress
!ip-address
- Show the server's IP address
- Show the server's Ports: 2302, 2305
- Explain how to connect to the server using this info: Direct Connect

!time
!timezone
!timezones
- Post a list of current times in select timezones.
- Timezones: UTC, PST, MST, Arizona (they do not follow DST), CST, EST, WET, CET
- Mention the difference of hours from UTC per timezone

!mods
!mod
!modlist
!modlists
!mod-list
!mod-lists
- Point to #mod-list
- Explain how to find the right mods for an operation
- Explain how to download the mod preset
- Explain how to install the mod preset using the game launcher

!teamspeak
!teamspeak3
!ts
!ts3
- Point to #teamspeak-info
- Provide a link to Teamspeak 3's website
- Mention the IP address

!acre
!acre2
!radio
- Point to #acre2-info
- Provide a link to ACRE2's Steam Workshop page

!tfar
!tfarvsacre
!acrevstfar
- Explain why we do not use tfar. The explanation is the following:
"ACRE2 is a mature, radio communication mod that fulfills all the same roles that TFAR offers. It is also more user-friendly to install as it automatically tries to install to one's Teamspeak 3 client. It is also more realistic than TFAR and works off an entire infrastructure designed around it. SESO is not planning to switch radio communication mods in the foreseeable future."

!training
!tactics
!manual
- Point to #manuals
- Explain Operation Standard. See below:
"Operation Standard is a document for Engagement Hosts and Operation Hosts. It contains an introductory look on how to get started into hosting. It also contains common pitfalls and suggestions when designing missions for SESO. Anyone is allowed to look at it and use it for any purpose."
- Explain Tactics Training. See below:
"Tactics Training is a training document for all operators in SESO. It covers a wealth of knowledge about common sense infantry and combined arms tactics of the SESO unit. It also covers ROE, COC, and other considerations. Anyone is allowed to look at it and use it for any purpose."

!emblem
!unitemblem
!logo
!unitlogo
!patch
!unitpatch
- Point to #ingame-emblem
- Explain what is the purpose of that channel

!rules
!unitrules
!roles
!unitroles
- Point to #unit-rules
- Briefly explain what is in this channel
- Mention Wheaton's Rule. That is the most important thing to get from the channel. Make it link to http://www.wheatonslaw.com/

!discord
!tutorial
!howtouse
!new
- List all commonly used channels/categories and how to use them. See below to get an idea, feel free to change:

Bulletin Board
#announcements - Contains unit-wide announcements and non-Arma events
#engagement-announcements - Contains Arma 3 engagements from hosts such as Operations Hosts, and Engagements Hosts. Only pings Operators so make sure to get the right role in #2
#dev-announcements - Contains Bohemia Interactive's updates about the Arma franchise's development
Tutorials & Info
#mod-list - Contains mod presets for various operations and missions
#teamspeak-info - Tutorial on setting up Teamspeak 3 for SESO
#ingame-emblem - Tutorial on setting up a SESO emblem on your in-game avatar
#acre2-info - Tutorial on setting up ACRE2
#manuals - In-depth documents about best practices
General Text Channels
#introduce-yourself - Channel for newcomers
#general - General text chat. Feel free to open thread here
#nsfw-memes-no-porn - Channel to post one's memes. Does not have to be NSFW, can be SFW too
#screenshots-n-videos - Post Arma 3 media here
#loa - Report LOA (Leave of Absence) for when you are away for any period of time
#suggestions - Suggest anything to Operations Command regarding SESO
#operation-feedback - Provide feedback for host's operations and engagements
#bot-commands - Command the Discord bots here
General Voice Channels

Contain voice channels for casual use

Acre is not more mature. It's not more realistic. TFAR you can actually switch freqs and all of that. Not just freq 1, 2 , 3 etc.
you can use freq 121.5 (which is the general emergency freq btw) and all custom freqs.

I was about to argue with you about this until I researched into TFAR. It has come a long way since a few years ago. I guess they're also mature and realistic now.
I'll update the response in !tfar after the command is implemented. I'll just cut it down to "It's easier/faster to install on Teamspeak than TFAR." That one piece of TFAR still hasn't changed.

help ip still gives "The IP command gives the User the IP address, port, and a short tutorial on how to join the Arma 3 Server."

Funny enough, since this issue has been created, many commands have to be updated with new links pointing to the SESO Wiki. Not a big deal, I will implement it. The original issue has been met.