How to run Llama 4 with vLLM and SGLang

This guide covers serving the Llama 4 family. Sizing and launch commands per model appear below.

Engine support

The Llama 4 Scout Instruct card documents vLLM (OpenAI-compatible server) and SGLang, and says both handle multimodal input of text plus images. It requires Transformers 4.51.0 or higher. The card does not cover llama.cpp or Ollama, so check those projects directly.

Multimodal input

• Input is multilingual text and images, with output as text and code.
• The card says the model was tested with up to 5 input images. Meta tells developers to run their own testing beyond that.

Context length

The card lists a 10M token context window for Scout and 1M for Maverick. Set the server context limit (for example --max-model-len) to what your workload needs rather than the full window.

Tool calling

vLLM documents the llama4_pythonic parser for all Llama 4 models, used with its tool_chat_template_llama4_pythonic.jinja template. Unlike Llama 3, vLLM states that parallel tool calls are supported for Llama 4.

Quantization

The card provides BF16 weights and says int4 quantization is available on the fly. FP8 weights are listed for Maverick (see FP8).