What is CUTLASS?
CUTLASS (CUDA Templates for Linear Algebra Subroutines and Solvers) is NVIDIA's open-source collection of C++ template building blocks for writing high-performance linear algebra kernels directly in CUDA. Like cuBLAS, the name nods to the classic BLAS (Basic Linear Algebra Subprograms) standard, but the two solve different problems. cuBLAS is a closed library of ready-to-call routines: you hand it matrices and it picks a kernel internally. CUTLASS is a toolkit for building your own kernels out of reusable pieces, and it is aimed squarely at the third level of the BLAS hierarchy, general matrix multiplications (GEMMs).
The "templates" in the name are C++ templates, the language's mechanism for parametric polymorphism: the same abstraction compiles into specialized code for whatever data type, tile size, or architecture generation you target, without hand-duplicating the logic for each case.
Underneath modern CUTLASS sits CuTe, a library of Layout and Tensor types for describing how data and threads are arranged and moved, composably rather than through fixed loop structures. On top of CuTe, CUTLASS exposes a header-only library operating at three levels: the whole device, a single kernel, or a collective of threads (typically a thread block). At the collective level, a matrix multiply is conventionally split into a "mainloop" (the core tiling algorithm) and an "epilogue" (post-processing like scale factors or activation functions).
CUTLASS is how most of the fastest published matrix-multiply kernels on recent Streaming Multiprocessor generations get their Tensor Core utilization close to peak; hand-tuning that last stretch of performance without a framework like this is a specialist skill. The library ships with many reference kernel implementations that the broader open-source kernel-engineering community treats as a starting point, and Colfax International's public tutorials on CUTLASS internals are a well-regarded way into the material, though the C++ template metaprogramming underneath is genuinely dense.
If you would rather skip writing CUDA kernels altogether, cuBLAS covers most common GEMM shapes out of the box, and Aquanode's marketplace gives you H100, H200, B200, and MI300X instances with a full CUDA toolchain preinstalled to build and benchmark CUTLASS kernels on directly.
Building on GPUs? Aquanode runs the workload.
Deploy on H100, H200, B200, A100 and MI300X across a multi-provider marketplace, without racking your own hardware or committing to one cloud's spec sheet.
See also
cuBLAS
cuBLAS is NVIDIA's tuned, closed-source implementation of the BLAS standard for GPUs, and the default engine under most matrix math in PyTorch and similar frameworks. The column-major memory layout quirk that trips up almost everyone who calls it directly.
Tensor Core
A Tensor Core is the GPU hardware unit that executes an entire matrix multiply-accumulate as one instruction instead of one scalar multiply at a time. How that trade unlocks NVIDIA's highest FLOP counts, and why an H100 has only four of them per SM.
Kernel
A CUDA kernel is the function a GPU programmer writes and launches, executed once per thread across thousands of threads at once. How kernels map onto the thread and memory hierarchy, with two worked matrix-multiply examples.
