ROCm vs CUDA: What Actually Works on AMD in 2026

You have a training or inference job that runs on NVIDIA, and someone just offered you AMD Instinct capacity for less money or with more memory per GPU. The question isn't "does AMD make good hardware" — it does. The question is whether your specific stack — your framework version, your attention kernel, your quantization library — runs on it without a rewrite. That answer is different for a stock PyTorch training loop than it is for a pipeline built on xformers and 4-bit bitsandbytes, and most "ROCm vs CUDA" posts don't separate the two.

TL;DR: PyTorch ships official stable wheels for ROCm (pinned to the ROCm 7.2 ABI as of August 2026), and AMD's compatibility matrix lists MI300X, MI325X, and MI355X as supported hardware. Framework-level code — a standard PyTorch or vLLM training/inference loop — mostly just works. Where ROCm still lags: flash-attention has no native support in the upstream Dao-AILab repo (AMD maintains a separate fork), bitsandbytes and xformers are both officially classified "experimental" on ROCm by their own maintainers, and custom CUDA kernels need a manual HIPIFY pass with no guaranteed 1:1 translation. If your workload is a mainstream framework call, budget for AMD. If it's a hand-written CUDA kernel or a library that says "experimental" next to ROCm, budget for verification time first.

What HIP actually is

CUDA is NVIDIA's proprietary programming model — the API, the compiler, the kernel language are all NVIDIA's. AMD's answer is HIP (Heterogeneous-compute Interface for Portability), a C++ runtime API and kernel language that's deliberately close to CUDA's syntax, described by AMD as letting you write single-source code that runs on both AMD and NVIDIA hardware from a common code path (AMD HIP docs).

The migration tool is HIPIFY — "a ROCm tool to help developers migrate GPU programming from NVIDIA's CUDA language to AMD's HIP C++ programming language" (AMD HIPIFY docs). There are two variants:

• hipify-clang — a clang-based tool that actually parses the CUDA source and does semantic translation of API calls and kernel launches. Slower to run, but more accurate and better at flagging what it can't translate.
• hipify-perl — a lighter regex-based script (auto-generated from hipify-clang's pattern list) that does textual find-and-replace on CUDA API calls. Faster, cruder, less error detection.

AMD's own porting guide is explicit that HIPIFY handles the mechanical part — API calls, kernel launch syntax — not the whole job. It does not translate proprietary CUDA libraries that have no ROCm equivalent, and it won't performance-tune a kernel that was hand-tuned for NVIDIA's memory hierarchy (HIP porting guide). AMD's own recommendation is to start the port on a CUDA machine — since HIP compiles to CUDA there too — and get the translated code building and passing correctness checks before you ever touch AMD hardware. Nobody publishes a reliable percentage for "how much of your codebase HIPIFY will translate cleanly" — it depends entirely on how much of your code touches a CUDA-only library versus standard kernel patterns. Don't trust a number if you see one; there isn't an official one.

Does PyTorch actually work on ROCm?

Yes, as a first-class, officially distributed build — this is the single biggest thing that's changed in the last two years. As of August 2026, PyTorch's stable release (2.13.0) ships official Linux wheels built against ROCm, installable directly from pytorch.org:

pip3 install torch torchvision --index-url https://download.pytorch.org/whl/rocm7.2

That command pins to the ROCm 7.2 wheel ABI, which is one release line behind AMD's current stable ROCm (7.14.0, released July 16, 2026) — PyTorch's nightly builds track rocm7.14, but stable is still on the 7.2 line as of this writing (pytorch.org install matrix, checked August 23, 2026). If you need the newest ROCm line, you're on nightly PyTorch, not stable — worth knowing before you plan a production pin.

AMD's ROCm compatibility matrix (checked August 23, 2026) lists the current supported hardware as the full Instinct line — MI355X, MI350X, MI350P, MI325X, MI300X, MI300A, MI250X/MI250, MI210, MI100 — plus Radeon RX 9000/7000-series and select Ryzen APUs, across Ubuntu, RHEL, Debian, SLES, and WSL2 (support windows vary by GPU generation). If your target is an MI300X on a documented OS, you're running a configuration AMD explicitly supports, not a community workaround.

Where the ecosystem still bites

This is the part that gets glossed over. "PyTorch works" doesn't mean "every library your PyTorch code imports works." Here's the honest state of the libraries people actually build on, checked against each project's own docs.

The pattern in that table is consistent: anything that's a mainstream framework maintained by a large team with AMD's direct involvement (PyTorch, Triton, vLLM, DeepSpeed, JAX) is officially supported and reasonably current. Anything that started as a hand-tuned CUDA kernel library from a smaller team (flash-attention, bitsandbytes, xformers) exists on ROCm only through a fork or an "experimental" label from the maintainer, and you should treat it as something to test before you commit, not something to assume.

flash-attention is the sharpest example. The most-used implementation for fast attention in modern transformer stacks has no merged AMD support in its own upstream repo — three years after the request was opened. That's not a knock on AMD; it reflects a small maintainer team prioritizing their own hardware. But it means "flash-attention on ROCm" always means "AMD's fork," with its own release cadence and its own gaps (sliding-window attention is documented as work-in-progress on AMD's side, per AMD's own model acceleration library docs) — not "the same flash-attention you already use, recompiled."

The MI300X memory argument, and when it flips the decision

The reason people put up with the ecosystem gaps at all is memory. AMD's MI300X product page states 192GB of HBM3 at 5.325 TB/s of bandwidth. Compare that to NVIDIA's own published specs: the H100 SXM carries 80GB HBM3 at 3.35 TB/s, and the H200 carries 141GB HBM3e at 4.8 TB/s. MI300X has more memory and more bandwidth than either, on a single card. You can see current on-demand rates for it, along with every other card we carry, on our MI300X page and the full GPU index.

That gap matters most for one specific case: a model that doesn't fit on a single H100 and would otherwise need tensor-parallel splitting across two or more GPUs just to load. A 70B model in FP16 needs roughly 140GB just for weights before you account for KV cache — that doesn't fit on one H100 (80GB) or even one H200 (141GB, and you'd want headroom for context). It fits comfortably on one MI300X. If your bottleneck is "the model doesn't fit," MI300X changes what's possible on a single GPU, not just what's cheaper. If your bottleneck is compute throughput on a model that already fits fine on an H100, the memory advantage doesn't move the needle, and you're back to the ecosystem question above deciding the outcome.

We cover the inference performance side of that trade-off with real vLLM and SemiAnalysis benchmark numbers separately in MI300X vs H100 vs H200 for inference, and the ComfyUI/image-generation-specific compatibility picture in ComfyUI on AMD — both go deeper than a spec-sheet comparison.

Inference vs. training maturity

Inference is the more mature ROCm story right now. vLLM and SGLang both get regular AMD-specific kernel work, AMD ships versioned Docker images tuned for MI300X/MI325X, and a standard serve-a-model-with-vLLM workflow is close to a drop-in swap from NVIDIA. Training is where the gaps concentrate: distributed training frameworks (DeepSpeed, Megatron-style pipelines) have documented feature gaps like the missing gradient-accumulation fusion noted above, and any training run that leans on a hand-written CUDA kernel for a custom layer inherits the full HIPIFY-and-verify burden with no compiled kernel to fall back to.

Frequently asked questions

Does PyTorch work on AMD GPUs?

Yes. PyTorch's stable release ships official Linux wheels built against ROCm (pinned to the ROCm 7.2 ABI as of August 2026, with nightlies tracking AMD's newer 7.14 line), installable directly from pytorch.org with no source build required.

What is HIP and how is it different from CUDA?

HIP (Heterogeneous-compute Interface for Portability) is AMD's C++ GPU programming API and kernel language, deliberately close to CUDA's syntax so code can be translated with the HIPIFY tool and compiled to run on either AMD or NVIDIA hardware. It is not CUDA itself — it's AMD's own runtime that CUDA-like code gets ported to.

Does flash-attention run on AMD?

Not through the original upstream repository — Dao-AILab's flash-attention has an open, unmerged feature request for AMD support dating to December 2023. AMD maintains a separate fork (ROCm/flash-attention) with CK and Triton backends that covers MI300X and newer Instinct hardware, but it's a distinct codebase with its own gaps, not the same package recompiled.

Is bitsandbytes supported on ROCm?

Hugging Face, bitsandbytes' own maintainer, classifies ROCm support as experimental rather than official. Pre-built wheels exist only for specific ROCm versions and GPU architectures; anything outside that combination requires building from source.

When does AMD's memory advantage matter most?

When a model doesn't fit on a single NVIDIA GPU without tensor-parallel splitting — roughly, models in the 65-100B-parameter range at FP16 that need more than 80-141GB. MI300X's 192GB fits many of those on one card. If your model already fits comfortably on an H100 or H200, the memory advantage doesn't change your decision, and library compatibility becomes the deciding factor instead.

Sources

• AMD HIP documentation, checked August 23, 2026.
• AMD HIPIFY documentation, checked August 23, 2026.
• AMD HIP porting guide, checked August 23, 2026.
• PyTorch install matrix, checked August 23, 2026 — ROCm 7.2 stable wheel pin.
• AMD ROCm compatibility matrix, checked August 23, 2026 — ROCm 7.14.0, released July 16, 2026.
• AMD Triton install docs, checked August 23, 2026.
• vLLM GPU installation docs, checked August 23, 2026.
• DeepSpeed getting started docs, checked August 23, 2026.
• AMD JAX install docs, checked August 23, 2026.
• Dao-AILab/flash-attention#707, open since December 2023, checked August 23, 2026.
• ROCm/flash-attention, checked August 23, 2026.
• AMD model acceleration library docs, checked August 23, 2026 — sliding-window attention noted as work-in-progress.
• Hugging Face bitsandbytes installation docs, checked August 23, 2026.
• facebookresearch/xformers and ROCm/xformers, checked August 23, 2026.
• AMD MI300X product page, checked August 23, 2026 — 192GB HBM3, 5.325 TB/s.
• NVIDIA H100 product page, checked August 23, 2026 — 80GB HBM3, 3.35 TB/s.
• NVIDIA H200 product page, checked August 23, 2026 — 141GB HBM3e, 4.8 TB/s.