"H100 specs" sounds like it should be one number, but NVIDIA sells three physically different H100 cards under that name, and mixing them up is the most common mistake in every spec table you'll find online. This post is the reference: every variant, every verified number, what Hopper's architecture actually changed, and where an H100 fits against A100 and H200. If you're here to check current rental prices instead, that's a separate, faster-moving page — see rent an H100.
TL;DR: The H100 ships in three variants: SXM5 (80GB HBM3, 3.35 TB/s bandwidth, up to 700W, NVLink 900GB/s — the fastest, used in DGX/HGX servers), PCIe (80GB HBM2e, 2.0 TB/s, 350W default, standard servers), and NVL (a dual-GPU board, 94GB HBM3 per GPU / 188GB combined, 3.9 TB/s per GPU, aimed at large-model inference). All three run on the Hopper architecture with fourth-generation Tensor Cores, FP8 support via the Transformer Engine, and 80 billion transistors on TSMC's 4N process. An 80GB H100 fits a 70B-parameter model in FP8/INT8 with room for KV cache, but not comfortably in FP16 alongside a large batch — that's the gap H200's 141GB closes.
The three H100s, side by side
NVIDIA's own product specifications, from the H100 datasheet and H100 PCIe product brief:
| Spec | H100 SXM5 | H100 PCIe | H100 NVL |
|---|---|---|---|
| Memory | 80 GB HBM3 | 80 GB HBM2e | 94 GB HBM3 per GPU (188 GB per dual-GPU board) |
| Memory bandwidth | 3.35 TB/s | 2.0 TB/s | 3.9 TB/s per GPU |
| FP64 Tensor Core | 67 TFLOPS | ~48 TFLOPS | 60 TFLOPS |
| TF32 Tensor Core (dense) | 989 TFLOPS | ~756 TFLOPS | 835 TFLOPS |
| FP16/BF16 Tensor Core (dense) | 1,979 TFLOPS | ~1,513 TFLOPS | 1,671 TFLOPS |
| FP8 Tensor Core (dense) | 3,958 TFLOPS | ~3,026 TFLOPS | 3,341 TFLOPS |
| NVLink bandwidth | 900 GB/s | 900 GB/s (via bridge, 2-GPU only) | 600 GB/s |
| PCIe interface | Gen5 x16 | Gen5 x16 | Gen5 x16 |
| TDP | Up to 700 W (configurable) | 350 W default (200-600 W range) | 350-400 W per GPU |
| Form factor | SXM5 (server-integrated) | Dual-slot FHFL, air-cooled | Dual-slot PCIe, air-cooled |
| MIG | 7 instances | 7 instances | 7 instances (per GPU) |
A note on PCIe's TFLOPS: NVIDIA's H100 PCIe product brief lists FP32 and INT8 explicitly, but not every dense Tensor Core figure by name — the ~48/756/1,513/3,026 TFLOPS figures above are the PCIe/SXM5 ratio (roughly 76.5%) applied consistently across NVIDIA's published SXM5 numbers, which is the standard way vendors derive the PCIe column since PCIe runs the same GH100 die at a lower clock and power envelope. Treat these four PCIe figures as close estimates rather than a directly-published NVIDIA number; the memory, bandwidth, TDP, and form-factor rows above are all pulled directly from NVIDIA's PCIe product brief.
The practical difference: SXM5 is the fastest and highest-bandwidth version, but it only exists inside NVIDIA's own server platforms (DGX H100, HGX H100) — you can't buy a standalone SXM5 card for an arbitrary server. PCIe is the version that fits a standard server and the one most cloud providers rent as a single-GPU instance. NVL is a two-GPU board built specifically for LLM inference workloads that need more memory than a single 80GB card but don't need a full 8-GPU DGX system.
What Hopper actually added over Ampere
The generation before H100 was A100, built on the Ampere architecture. Hopper's changes aren't just "more of the same, faster" — several are structural:
- Transformer Engine. Hopper's fourth-generation Tensor Cores include dedicated logic that dynamically manages FP8 and FP16 precision per layer during training, keeping accuracy close to FP16 while running at FP8 throughput. A100 has no equivalent — its Tensor Cores top out at TF32/FP16/INT8 with no automatic per-layer precision management.
- FP8 support. H100 doubles peak throughput over FP16/BF16 by adding native FP8 Tensor Core math, roughly 4x A100's FP16 dense throughput on paper. This is the single biggest reason H100 outperforms A100 on modern transformer training and inference.
- DPX instructions. New instructions that accelerate dynamic programming algorithms (used in genomics, route optimization, and some graph workloads) — an A100 has no equivalent instruction set for these.
- Thread block clusters. A new level of the CUDA execution hierarchy above thread blocks, letting SMs cooperate and share data (via a feature called distributed shared memory) across a cluster rather than just within one block — this gives Hopper kernels a way to coordinate work at a scale Ampere's programming model didn't expose.
- Higher NVLink bandwidth. 900 GB/s on H100 SXM5 vs 600 GB/s on A100 SXM, meaningfully faster for multi-GPU training that shards a model across cards.
None of this means A100 is obsolete — for workloads that fit comfortably in 40-80GB and don't need FP8, A100 remains price-efficient. But for anything transformer-shaped and throughput-sensitive, Hopper's FP8 path is the reason H100 isn't just an incremental bump.
What actually fits in 80GB
The number people actually want when they search "H100 VRAM" is: what model size does that get you. Rough rule of thumb, using 2 bytes/parameter for FP16 and 1 byte/parameter for FP8/INT8, before accounting for KV cache and activation memory:
| Precision | Model size that roughly fits in 80GB | Notes |
|---|---|---|
| FP16/BF16 | ~35-40B parameters | Leaves headroom for KV cache and a modest batch |
| FP8/INT8 | ~65-70B parameters | Standard way to run Llama-class 70B models on a single H100 |
| FP16, large batch or long context | Falls short of 70B | KV cache grows with context length and concurrency, eating into the same 80GB |
This is why H100 NVL (188GB combined across two GPUs) and H200 (141GB single-GPU) exist — they're both responses to the same problem, that 80GB is tight for the current generation of large open models once you add real batch sizes or long context windows. If your workload is VRAM-bound rather than compute-bound, that's the number to plan around before you pick hardware — see our full VRAM sizing breakdown for the exact formulas behind these estimates, including KV cache growth with context length and batch size.
H100 vs A100 vs H200 at a glance
| A100 (80GB) | H100 SXM5 | H200 SXM | |
|---|---|---|---|
| Architecture | Ampere | Hopper | Hopper (same die as H100) |
| Memory | 80 GB HBM2e | 80 GB HBM3 | 141 GB HBM3e |
| Memory bandwidth | 2.0 TB/s | 3.35 TB/s | 4.8 TB/s |
| FP16 Tensor Core (dense) | 312 TFLOPS | 1,979 TFLOPS | 1,979 TFLOPS |
| FP8 Tensor Core | Not supported | 3,958 TFLOPS | 3,958 TFLOPS |
| NVLink bandwidth | 600 GB/s | 900 GB/s | 900 GB/s |
H200 is worth calling out specifically: it uses the same GH100 compute die as H100 SXM5, so compute throughput is identical — the entire upgrade is memory capacity (141GB vs 80GB) and bandwidth (4.8 TB/s vs 3.35 TB/s). If your bottleneck is compute, H100 and H200 perform the same; if your bottleneck is memory (long context, large batch, bigger models), H200 is the direct answer. We cover this tradeoff in more depth in From A100 to H200, and the full A100 and H200 pages have current specs and pricing for each.
DGX H100: the full system
For teams evaluating buy-vs-rent at cluster scale, it's worth knowing what an 8-GPU DGX H100 system actually contains, per NVIDIA's own DGX H100 documentation: 8x H100 SXM5 GPUs (640GB total GPU memory), connected through 4x NVSwitches delivering 7.2 TB/s of GPU-to-GPU bandwidth, dual x86 CPUs, 2TB system memory, and 10x ConnectX-7 400Gb/s network interfaces for 1 TB/s of peak network bandwidth. System-level power draw runs up to roughly 10.2 kW. That power and cooling footprint is the practical reason most teams rent H100 capacity by the hour rather than buying and operating a DGX system themselves, unless they're running it near-continuously at high utilization.
Buying vs renting
An H100 SXM5 card retails in the tens of thousands of dollars, and a full 8-GPU DGX H100 system runs well into six figures before power, cooling, networking, and operations are added — figures that shift with supply and aren't worth quoting precisely here since list and street prices move independently. For a team that needs H100 capacity for a finite training run, a burst of inference load, or anything short of near-continuous multi-year utilization, renting is the default rational choice: no capital outlay, no facility power/cooling build-out, and the ability to switch GPU generation or provider when pricing or availability shifts.
We deliberately don't duplicate a live price table here — H100 rental rates move by the week across providers, and this post's job is the spec sheet, not the price sheet. For current H100 rates across providers, go to /gpu/h100; for pricing on every GPU model side by side, the live GPU Availability Index tracks lowest and median $/hr per model, and the GPU recommender tool can match your VRAM and throughput needs to the right card if you're not sure H100 is the right pick versus A100 or H200.
Frequently asked questions
How much VRAM does an H100 have?
80GB on both the SXM5 and PCIe variants. The H100 NVL variant, a dual-GPU board, carries 94GB per GPU (188GB combined). SXM5 uses faster HBM3 at 3.35 TB/s; PCIe uses HBM2e at 2.0 TB/s — same capacity, different memory technology and bandwidth.
What's the difference between H100 SXM and H100 PCIe?
SXM5 is the higher-performance variant, integrated into NVIDIA's own server platforms (DGX/HGX H100) with faster HBM3 memory (3.35 TB/s) and up to 700W TDP. PCIe is a standard add-in card for conventional servers, using HBM2e (2.0 TB/s) at a lower 350W default TDP. SXM5 is faster; PCIe is what fits in general-purpose infrastructure and is what most cloud providers rent as a single GPU.
What is H100 NVL for?
H100 NVL is a dual-GPU PCIe board built specifically for large-model inference, combining 188GB of memory across the pair (94GB per GPU) with 600 GB/s of NVLink between them. It targets models too large to serve comfortably on a single 80GB H100 without needing a full multi-GPU DGX system.
Is the H100 still worth buying/renting in 2026, or should I use H200?
If your workload is compute-bound and fits comfortably in 80GB, H100 and H200 perform identically since they share the same compute die — H100 is typically the cheaper rental option. If your workload is memory-bound (long context, large batch sizes, models pushing past ~70B parameters), H200's 141GB and 4.8 TB/s bandwidth remove a real bottleneck H100 doesn't solve.
What does a DGX H100 system cost to run?
NVIDIA's DGX H100 documentation lists system power draw up to roughly 10.2 kW for the 8-GPU configuration, before facility cooling overhead. That power and cooling requirement, combined with six-figure-plus hardware cost, is why most teams below hyperscale rent H100 capacity by the hour instead of buying a DGX system outright.
