What is nvidia-smi?

nvidia-smi, short for NVIDIA System Management Interface, is the command line tool most people reach for first when they want to know what's actually happening on an NVIDIA GPU. It doesn't talk to the hardware directly; it's a thin front end over NVIDIA's Management Library (NVML), the same library that monitoring agents, cluster schedulers, and dashboards call under the hood to read GPU state.

Run it with no arguments and it prints one snapshot table with a row per GPU: model name, driver and CUDA versions, a PCI bus ID, fan speed and temperature, current power draw against the card's power cap, a GPU utilization percentage, and how much of GPU RAM is currently allocated. On a box with several idle GPUs, that table looks something like this:

+-----------------------------------------------------------------------------------------+
| NVIDIA-SMI 580.95.05             Driver Version: 580.95.05     CUDA Version: 13.0       |
|-----------------------------------------+------------------------+----------------------+
| GPU  Name                 Persistence-M | Bus-Id          Disp.A | Volatile Uncorr. ECC |
| Fan  Temp   Perf          Pwr:Usage/Cap |           Memory-Usage | GPU-Util  Compute M. |
|                                          |                        |               MIG M. |
|==========================================+========================+======================|
|   0  NVIDIA B200               On       |   00000000:51:00.0 Off |                    0 |
| N/A   27C    P0             136W / 1000W|      0MiB /  183359MiB |      0%      Default |
|                                          |                        |             Disabled |
+-----------------------------------------+------------------------+----------------------+

Anyone who has SSH'd into a training box has seen a table like this dozens of times; it's the closest thing GPU computing has to a universal status readout.

Past that summary view, nvidia-smi can list every process currently holding a handle to a GPU (via -q, --query, or pmon), which tends to be the fastest way to find out which job on a shared box is actually holding memory. It also works as a lightweight control surface: operators use it to toggle persistence mode (-pm) so the driver stays resident between short jobs instead of reinitializing each time, set compute mode (-c) to decide whether a GPU accepts one process exclusively or several concurrently, cap power draw (-pl), pin application or memory/graphics clocks (-ac, -lgc, -lmc), and force a reset (-r) on a GPU a crashed process left in a bad state.

Output can be printed as plain text or as XML (-x) for a script to parse. The important caveat is that the text layout is not a stable interface: NVIDIA can and does change column names and spacing between driver releases. Anything that needs to parse GPU state reliably in production should link against the NVML C library directly, or use its official Python bindings, rather than scraping nvidia-smi's stdout.

Those same utilization and memory figures are also the raw input most autoscalers and bin-packing schedulers use to decide whether a GPU has room for one more job, which is part of why hardware-accelerated GPU scheduling and driver-level queueing behavior affect how trustworthy an nvidia-smi snapshot is from one instant to the next.

Aquanode's GPU metrics feature reads the same NVML data nvidia-smi does, so you can watch utilization, memory, and power across a fleet of running instances, across whatever GPU architecture generation you've rented, without opening a shell and typing the command by hand.