LLM
How to deploy NVIDIA-Nemotron-3-Ultra-550B-A55B-BF16 on a GPU cloud
A 550B (MoE) language model for chat and agentic tool use. Full specs, license and use cases.
NVIDIA-Nemotron-3-Ultra-550B-A55B-BF16 size and hardware requirements
560.5B
Total parameters
Mixture-of-experts: 22 of 512 experts active per token (~55B active parameters, per the model's own "A55B" name)
Active parameters
BF16
Published precision
1252.9 GB
Min VRAM (native)
| Precision | Weight size on disk | Required VRAM | Cheapest live fit | GPUs needed | Est. $/hr (full fit) |
|---|---|---|---|---|---|
| BF16 | 1044.1 GB | 1252.9 GB | No capable live offer found | – | – |
| FP8 (quantized) | 522.0 GB | 626.4 GB | RTX PRO 6000 | 7 | $11.48/hr |
| INT4 (quantized) | 261.0 GB | 313.2 GB | RTX A6000 | 7 | $2.31/hr |
How to run NVIDIA-Nemotron-3-Ultra-550B-A55B-BF16
Run NVIDIA-Nemotron-3-Ultra-550B-A55B-BF16 with vLLM
From nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-BF16's own model card (8x GPU single-node recipe, trimmed).
docker run -d --name nemotron-ultra-vllm \
--gpus all \
--ipc=host \
--network=host \
-v $MODEL_CKPT:/model:ro \
vllm/vllm-openai:v0.22.0 \
/model \
--host 0.0.0.0 \
--port 8000 \
--served-model-name nvidia/nemotron-3-ultra \
--trust-remote-code \
--tensor-parallel-size 8 \
--enable-expert-parallel \
--dtype bfloat16 \
--max-model-len 262144Source: https://huggingface.co/nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-BF16/raw/main/README.md
Deploy NVIDIA-Nemotron-3-Ultra-550B-A55B-BF16 on Aquanode
Aquanode has no one-click deploy template for NVIDIA-Nemotron-3-Ultra-550B-A55B-BF16; you install the inference engine yourself with the commands below. Aquanode sells GPU pods billed per second, not a hosted inference API.
- Launch a bare GPU pod sized to the requirement above (1253 GB VRAM or more).
- Open a terminal on the pod, or save one of the commands above as a startup script so it runs automatically the first time the pod boots.
- Run the command and connect to the resulting endpoint.
Launch a GPU pod