What is the roofline model?

The roofline model is a small, visual way of answering one question about a kernel: is it limited by how fast the hardware can move data, or by how fast it can do arithmetic on that data? Rather than requiring a full simulation, it plots a single point derived from the kernel against two hardware-derived ceilings, or "roofs," and the shape of that plot tells you immediately which limit applies.

The two roofs come straight from the target GPU's spec sheet. The "compute roof" is the peak rate of the hardware doing arithmetic, whether on CUDA Cores or Tensor Cores: its arithmetic bandwidth, in floating-point operations per second. The "memory roof" is the peak rate at which the GPU can move bytes between GPU RAM and the compute units: its memory bandwidth, in bytes per second. Both numbers depend on which execution unit you're asking about, not just which GPU, since Tensor Cores and CUDA Cores on the same die have very different peak throughput.

Plotting puts arithmetic intensity, operations performed per byte moved, on the x-axis, and achieved performance, in operations per second, on the y-axis. The compute roof is a flat horizontal line at the height of the arithmetic bandwidth. The memory roof is a line through the origin whose slope equals the memory bandwidth, since operations per second divided by operations per byte gives you bytes per second, exactly the units a bandwidth needs. A kernel's arithmetic intensity fixes its x-coordinate, and whichever roof sits lower at that x-coordinate is the one actually capping its throughput: under the flat roof means compute-bound, under the slanted one means memory-bound. In practice, few real kernels sit exactly on either roof, since overhead from launch latency, synchronization, and imperfect memory coalescing eats into the achievable ceiling.

Where the two roofs cross is called the ridge point, and its x-coordinate is the minimum arithmetic intensity a kernel needs to escape the memory roof and become compute-bound. A lower ridge point makes a system easier to run at peak efficiency, but because memory bandwidth has historically scaled more slowly than arithmetic throughput, ridge points have generally crept rightward across hardware generations, meaning newer accelerators demand higher arithmetic intensity before they stop being memory-bound. NVIDIA's Nsight Compute tool automates roofline analysis for profiled kernels directly, plotting real kernels against their GPU's roofs without manual arithmetic.

Part of what makes the model useful is how much it leaves out. It says nothing about latency, only about throughput ceilings, and that omission is deliberate rather than an oversight; a simpler model that captures the dominant bottleneck beats a complete one nobody can reason about at a glance.

The model was introduced by Samuel Williams, Andrew Waterman, and David Patterson in a 2008 paper, at a moment when several long-running hardware trends were converging. Patterson had already argued, in an influential 2004 paper, that "latency lags bandwidth": across compute, memory, and storage alike, bandwidth had historically improved quadratically for every linear gain in latency, pointing toward a future of throughput-oriented rather than latency-oriented systems. Separately, memory subsystems like DRAM and caches had been scaling far more slowly than compute for years, a gap Wulf and McKee named the "memory wall" in 1994. And by the early 2000s, Dennard scaling, which had let clock speeds rise at constant power, had ended because of fixed transistor leakage current, even as Moore's Law kept delivering more transistors per chip. With clock speed no longer a free lever, the industry's answer to more transistors and a fixed power budget was specialization: building chips out of components tuned for particular jobs, an approach documented at length in Hennessy and Patterson's discussion of the Pixel Visual Core image co-processor in Computer Architecture.

Read together, those trends pointed at exactly the kind of hardware GPUs turned out to be: throughput-oriented, memory-bandwidth-constrained, and only reaching peak performance on workloads with enough arithmetic intensity to keep the compute units fed. That's precisely why large matrix multiplications, the kind Tensor Cores are built for, need enough batching or reuse to actually earn a spot near the compute roof rather than stalling on the memory one.

Sizing arithmetic and memory bandwidth correctly for the GPU you're actually renting matters more than the roofline chart itself; Aquanode's GPU pages list both figures per model so you can work out a kernel's ridge point before you provision anything.