Deploy AI models on H100, A100, H200, and AMD MI300X GPUs with up to 40% cost savings. Lightning-fast machine learning inference on enterprise GPU infrastructure.
How much GPU memory a model needs at FP16, FP8 and INT4, and the cheapest live GPU that actually fits it. Pick a popular model or enter your own parameter count.
Required VRAM = weight size at that precision × 1.2 (KV-cache/activation overhead). A GPU is only matched if its hardware supports the row's compute dtype (see the GPU recommender's methodology).
How this is calculated
Required VRAM = parameter count × bytes-per-parameter at the chosen precision (FP32 = 4 bytes, BF16/FP16 = 2 bytes, FP8/ INT4-quantized = 1 or 0.5 bytes), × 1.2 to account for KV-cache, activation memory and allocator fragmentation during inference. That 1.2× is a documented rule-of-thumb for single-request serving at a moderate context length, not a simulator; see the full methodology on the models page. A GPU is only matched to a precision row if its hardware actually supports that compute dtype (FP8 and INT4-serving kernels require specific tensor-core generations), same rule used everywhere else on the site.
What fits on a specific GPU
Going the other direction: starting from a card's VRAM and asking what models fit in it:
