What is arithmetic intensity?

Arithmetic intensity measures how much computation a kernel does per byte of data it moves. Formally, it's the ratio of arithmetic operations (FLOPs) to bytes read from and written to memory over the life of a kernel launch. A kernel with high arithmetic intensity spends most of its time computing; a kernel with low arithmetic intensity spends most of its time waiting on loads and stores.

That ratio matters because on modern GPUs, arithmetic throughput has grown much faster across generations than memory bandwidth has. A kernel that doesn't do enough math per byte loaded sits idle waiting on GPU RAM, no matter how fast its cores are. The roofline model plots this trade-off directly, with arithmetic intensity on the x-axis: below a system-specific threshold a kernel is memory-bound, above it the kernel is compute-bound.

Because the two subsystems trade off against each other, one of the more useful optimization moves is deliberately shifting work from memory onto the arithmetic units, which raises intensity even though it adds total operations. Two examples show the pattern.

Compressing data before it's written to global memory, then decompressing it on read, cuts the bytes crossing the memory bus, at the cost of extra decompression math. If the kernel was memory-bound to start, this is close to a free win: fewer bytes moved, a higher ratio of FLOPs to bytes.

Gradient checkpointing in backpropagation works the same way. A naive backward pass keeps every forward-pass activation resident in global memory for reuse later, which is a lot of memory traffic. Checkpointing keeps only a fraction of those activations and recomputes the rest during the backward pass, trading memory bandwidth for arithmetic and raising intensity.

Arithmetic intensity doesn't stay fixed as a problem grows: operation count and memory footprint usually scale differently with problem size N, so intensity scales with N too. A kernel with O(1) arithmetic complexity and O(N) memory complexity gets less compute-friendly as N grows (intensity scales as O(1/N)); one with O(N) arithmetic and O(1) memory gets more compute-friendly (O(N) intensity). Three common kernels show the pattern:

Matrix multiplication's O(N) scaling stands out: its operation count grows as N³ while its memory footprint grows only as N². That favorable scaling is a big part of why neural networks, built mostly from stacked matrix multiplies, map so well onto GPUs: bigger matmuls get more compute-bound rather than less, the opposite of what happens to most kernels as they scale up.

How much arithmetic intensity a kernel needs before it stops being memory-bound is its ridge point: the intensity where the memory-bandwidth ceiling meets the arithmetic-throughput ceiling in the roofline model. It's a fixed property of a GPU and precision, computed once as arithmetic bandwidth over memory bandwidth:

Ridge points climb across both generations and lower precisions, since each new Tensor Core generation adds arithmetic throughput faster than it adds memory bandwidth. A kernel comfortably compute-bound on an A100 in BF16 can turn memory-bound on a B200 running FP8, simply because the arithmetic ceiling moved further than the memory ceiling did.

Matching a kernel to hardware mostly means comparing its arithmetic intensity to the ridge points above, rather than chasing the highest FLOPs number on a spec sheet. Aquanode's GPU recommender picks between H100, B200, and other listed SKUs by that shape, and the marketplace lets you rent whichever one clears your kernel's ridge point.