A TPU, or Tensor Processing Unit, is a chip Google designed for machine learning. Google's documentation calls TPUs "Google's custom-developed, application-specific integrated circuits (ASICs) used to accelerate machine learning workloads."
How it differs from a GPU
Google's TPU introduction describes TPUs as hardware designed for large matrix operations, with high-bandwidth memory on the chip. Code reaches them through the XLA compiler, which compiles the graph emitted by an ML framework.
The same page compares workload fit:
- CPUs suit quick prototyping and small models.
- GPUs suit models with many custom PyTorch or JAX operations, and medium to large models.
- TPUs suit models dominated by matrix computation, long training runs and large models with large batch sizes. Google says they are not suited to workloads needing high-precision arithmetic or frequent branching.
Where to go next
For how TPUs are reached, see how to access a Google TPU. For the architecture and cost comparison, read TPU vs GPU, and for the other chips in this space, NVIDIA's main competitors in AI.
