Instructions to use deepseek-ai/DeepSeek-R1-Distill-Qwen-32B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use deepseek-ai/DeepSeek-R1-Distill-Qwen-32B with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="deepseek-ai/DeepSeek-R1-Distill-Qwen-32B")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("deepseek-ai/DeepSeek-R1-Distill-Qwen-32B")
model = AutoModelForCausalLM.from_pretrained("deepseek-ai/DeepSeek-R1-Distill-Qwen-32B")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Inference
HuggingChat
Notebooks
Google Colab
Kaggle
Local Apps

vLLM

How to use deepseek-ai/DeepSeek-R1-Distill-Qwen-32B with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "deepseek-ai/DeepSeek-R1-Distill-Qwen-32B"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "deepseek-ai/DeepSeek-R1-Distill-Qwen-32B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/deepseek-ai/DeepSeek-R1-Distill-Qwen-32B

SGLang

How to use deepseek-ai/DeepSeek-R1-Distill-Qwen-32B with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "deepseek-ai/DeepSeek-R1-Distill-Qwen-32B" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "deepseek-ai/DeepSeek-R1-Distill-Qwen-32B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "deepseek-ai/DeepSeek-R1-Distill-Qwen-32B" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "deepseek-ai/DeepSeek-R1-Distill-Qwen-32B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use deepseek-ai/DeepSeek-R1-Distill-Qwen-32B with Docker Model Runner:
```
docker model run hf.co/deepseek-ai/DeepSeek-R1-Distill-Qwen-32B
```

comfyui-deepseek-r1

#15

by zwpython - opened Jan 23, 2025

Discussion

zwpython

Jan 23, 2025

https://github.com/ziwang-com/comfyui-deepseek-r1

fblgit

Jan 28, 2025

some ENG translations would be helpful for the audience. in any case u should contribute this to the comfy flows sites instead.

zwpython

Jan 28, 2025

https://github.com/ziwang-com/comfyui-deepseek-r1

comfyui-deepseek-r1 节点插件.Comfyui-deepseek-r1 Node Plugin
节点插件安装很简单，copy到comfyui的custom_nodes子目录下即可。
The installation of the node plugin is very simple, just copy it to the custom_nodes subdirectories of comfyui.
ps,2025-1-28
github搜了半天，好像是目前唯一支持comfyui本地化部署deepseek-r1的方案
官方没有提供，comfy主要aigc。其他或者ollama 或者api ,都挺啰嗦
After searching on GitHub for a while, it seems to be the only solution that currently supports localized deployment of DeepSeek-r1 on Comfyui
Officially not provided, comfy is mainly AIGC. Other options such as Olama or API are quite verbose

zwpython

Jan 28, 2025

一致性，deepseek能及格吗？
Consistency, can Deepseek pass?
https://mp.weixin.qq.com/s/udDhvZqM2SyQiDMtG9fF5g?token=1826715191&lang=zh_CN

硬核blog：一致性，deepseek能及格吗？
Hardcore Blog: Consistency, Can Deepseek Pass?
圈粉猛人无数，连华为前总裁都主动+粉。
There are countless fans in the circle, even the former president of Huawei actively gained followers.

deepseek-r1的成功，标志着人类ai、gpt、大模型，终于从野蛮的算力时代，过度到“逻辑”思维，时代。
The success of DeepSEEK-R1 signifies the emergence of human AI gpt、 The big model has finally transitioned from the barbaric era of computing power to the era of "logical" thinking.

参见：Refer to:

大模型的尽头，可能是logNet逻辑网络模型
The end of the big model may be the logNet logical network model

不懂一谈大模型=耍流氓
I don't understand. Talking about big models is like playing rogue

GPT刚问世时，全球震撼，不过一线的研发者却清晰地知道，这只是：
When GPT first came out, it was a global shock, but frontline developers knew clearly that this was just:

起点：
starting point:

真正的big thing是：一致性
The real big thing is consistency

AI时代，三个月迭代升级一次。
In the era of AI, there is an iterative upgrade every three months.

三年，差不多等于一个世纪。
Three years is almost equivalent to a century.

遗憾的是，三年过去了，一个世纪，过去了。
Unfortunately, three years have passed, a century has passed.

至今为止，无人成功。
So far, no one has succeeded.

如果说这个问题的最终答案是：1+1=2
If the final answer to this question is: 1+1=2

所有的AI巨头：openAI,谷歌、facebook，微软、grok
All AI giants: openAI, Google facebook， Microsoft grok

连方向，都还没找不到。
I haven't even found the direction yet.

可能依然在黑暗时代，苦苦摸索：10000+10000=？
Perhaps still in the dark ages, struggling to explore: 10000+10000=?

如今，试卷已经发到deepseek团队手上？
Has the exam paper been sent to the Deepseek team now?

问题是：需要多久，才能交卷？
The question is: How long will it take to submit the paper?

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment