A VAE (variational autoencoder) is the model that converts between images and the compressed latent space the diffusion model works in. ComfyUI's own description for the VAE output: "The VAE model used for encoding and decoding images to and from latent space."
Where the VAE comes from
- Inside the checkpoint. The Load Checkpoint node has three outputs: MODEL, CLIP and VAE. For many checkpoints, the VAE is bundled and you connect that output to the VAE Decode node.
- A separate file. Some models ship the VAE separately. Put it in
ComfyUI/models/vaeand add a Load VAE node, then connect its output to VAE Decode instead.
Your final image comes out of VAE Decode, so the VAE you pick affects the finished picture.
Memory note
The VAE step has its own memory cost. If it is what runs out of memory, --cpu-vae runs it on the CPU: see ComfyUI out of memory.
For the other folders, see where to put checkpoints in ComfyUI. For the tool as a whole, what ComfyUI is, and the ComfyUI custom node install guide for add-ons.
Sources
- ComfyUI nodes.py (CheckpointLoaderSimple, VAELoader). github.com
- ComfyUI folder_paths.py. github.com
- ComfyUI cli_args.py (--cpu-vae). github.com