GPU Glossary

Core

A GPU core is one of the compute units packed inside a Streaming Multiprocessor. What separates a CUDA Core from a Tensor Core, and why the CPU-core analogy is misleading.

CUDA Core

A CUDA Core is the unit inside a Streaming Multiprocessor that executes scalar arithmetic, one instruction issued to a whole group at a time. What separates it from a Tensor Core, whether more of them means a faster GPU, and where they fit in AI training and inference.

GPU RAM

GPU RAM is the large off-die memory pool every Streaming Multiprocessor shares, built from slower, denser DRAM cells rather than the SRAM used in registers and cache.

Graphics/GPU Processing Cluster (GPC)

A Graphics/GPU Processing Cluster (GPC) groups Texture Processing Clusters and a raster engine into one of the largest physical units on an NVIDIA die. What changed with H100-class thread block clusters.

Register File

The register file is the fastest, closest-to-the-core memory on a Streaming Multiprocessor. How it's organized, what backs it in PTX, and why using too much of it per thread quietly kills occupancy.

Streaming Multiprocessor (SM)

A Streaming Multiprocessor is the closest thing an NVIDIA GPU has to a CPU core, but far simpler and far more numerous. How SMs trade per-thread sophistication for massive parallelism and near-free context switching.

Tensor Core

A Tensor Core is the GPU hardware unit that executes an entire matrix multiply-accumulate as one instruction instead of one scalar multiply at a time. How that trade unlocks NVIDIA's highest FLOP counts, and why an H100 has only four of them per SM.

Warp Scheduler

The Warp Scheduler decides which warp of threads runs next on a Streaming Multiprocessor, every single clock cycle. Why that decision is nearly free on a GPU and expensive on a CPU.

CUDA Programming Model

The CUDA programming model organizes GPU code around a nested hierarchy of threads and memory. The three abstractions from NVIDIA's own programming guide, and why they let one program get faster on every new GPU without a rewrite.

Kernel

A CUDA kernel is the function a GPU programmer writes and launches, executed once per thread across thousands of threads at once. How kernels map onto the thread and memory hierarchy, with two worked matrix-multiply examples.

Shared Memory

Shared memory is the fast, on-chip pool of memory a CUDA thread block uses to avoid repeatedly hitting slower global memory. The standard load-compute-store pattern it enables, and where bank conflicts come from.

Thread Block

A CUDA thread block is the smallest unit of thread coordination a programmer directly controls, sitting between a kernel grid and a single thread. How blocks are sized and why they must run independently of each other.

Warp

A warp is a group of 32 threads that a GPU schedules and executes together in lockstep. What warp divergence costs, how warps enable latency hiding, and why they sit outside the official CUDA thread hierarchy.

cuBLAS

cuBLAS is NVIDIA's tuned, closed-source implementation of the BLAS standard for GPUs, and the default engine under most matrix math in PyTorch and similar frameworks. The column-major memory layout quirk that trips up almost everyone who calls it directly.

CUTLASS

CUTLASS is NVIDIA's open-source C++ template library for building custom high-performance GEMM kernels, the toolkit underneath many of the fastest published matrix-multiply implementations. How it differs from calling a fixed routine in cuBLAS.

Nsight Systems

NVIDIA Nsight Systems is a profiling and tracing tool for CUDA programs that lays CPU and GPU activity out on one timeline. Why almost every serious CUDA project ends up needing it.

nvidia-smi

nvidia-smi is the command line tool for querying and managing NVIDIA GPUs, built on the NVML management library. What it reports, what it can change, and why its text output isn't a stable interface.

Arithmetic Intensity

Arithmetic intensity is the ratio of compute operations to bytes moved in a kernel. Why it decides whether a workload is compute-bound or memory-bound, and how tricks like recomputation trade memory traffic for extra FLOPs.

Bank Conflict

A bank conflict is what happens when threads in a warp hit the same shared memory bank from different addresses, forcing the hardware to serialize accesses it could otherwise run in parallel.

Compute-bound

A compute-bound kernel is limited by arithmetic throughput rather than memory bandwidth. When LLM inference hits this regime, and a back-of-the-envelope estimate for the batch size it takes to get there.

Memory Coalescing

Memory coalescing is a hardware technique that folds several threads' logical memory reads into one physical DRAM access. Why it exists, how it maps onto a warp, and a benchmark showing what a bad access pattern costs.

Roofline Model

The roofline model plots a kernel's arithmetic intensity against two hardware ceilings, memory bandwidth and arithmetic bandwidth, to show at a glance whether it's compute-bound or memory-bound. Where it came from and why GPUs need it.

GPU Architecture

How NVIDIA GPUs are actually built, from Graphics Processing Clusters and Streaming Multiprocessors down to the memory hierarchy, and how Ampere, Hopper, and Blackwell differ.

NVLink vs InfiniBand

NVLink connects GPUs inside one server; InfiniBand connects servers to each other. Where the two interconnects overlap, where they don't, and why big clusters run both.

NVLink vs PCIe

NVLink and PCIe both move data in and out of a GPU, but at very different scales. Where each interconnect wins on bandwidth, latency, cost, and compatibility.

Hardware-Accelerated GPU Scheduling (HAGS)

Hardware-Accelerated GPU Scheduling (HAGS) lets a GPU manage its own memory and task queue instead of leaving that work to the OS kernel. What it changes and when it helps.