How to run DeepSeek-R1 with vLLM and SGLang

This guide covers serving the DeepSeek-R1 family. Sizing and launch commands per model appear below.

Engine support

The model card gives example commands for vLLM and SGLang, using a distilled model (DeepSeek-R1-Distill-Qwen-32B). The SGLang example passes --trust-remote-code. The card states no minimum engine versions and does not mention llama.cpp or Ollama.

Context length

DeepSeek-R1 and DeepSeek-R1-Zero support a 128K context window. The vLLM example on the card caps --max-model-len at 32768 for the distilled model, so raise it deliberately if you need more.

Prompting recommendations

The card lists these usage recommendations:

• Set temperature in the 0.5 to 0.7 range (0.6 recommended) to avoid repetition or incoherent output.
• Do not add a system prompt. Put all instructions in the user prompt.
• For math, add a directive such as "Please reason step by step, and put your final answer within \boxed."
• Force the response to begin with <think>\n so the model reasons thoroughly.

Distilled models

There are six distilled models: Qwen-based at 1.5B, 7B, 14B and 32B, and Llama-based at 8B and 70B. They are dense models that serve like their base architectures, so engine support follows the Qwen2.5 and Llama checkpoints they came from.

Pitfalls

• A system prompt can degrade results, which trips up OpenAI-style clients that always send one.
• Output contains the reasoning before the answer, so parse or strip the <think> section in your application.