How to fine-tune Gemma 3 (LoRA, QLoRA)

Gemma 3 spans 1B to 27B; the Gemma 3 hub lists sizes. For cost, see the training cost calculator and fine-tuning.

Checkpoint choice

The card offers pre-trained (PT) and instruction-tuned (IT) checkpoints. Start from PT for continued pre-training or a fully custom format, and from IT to keep chat behavior. Methods such as LoRA and QLoRA apply to both; the sources do not document target modules, so choose them from the layer names.

Data and template

• Train with the IT chat template. The card's examples use system and user roles.
• The launch post recommends bfloat16, which also suits training.
• The 1B model is text only. 4B and larger have a SigLIP vision tower at 896 x 896 input, so decide whether to freeze it or train only the language layers.
• The launch post mentions Unsloth integration for fine-tuning.

License limits on derivatives

Gemma 3 uses the Gemma Terms of Use, which you must accept on Hugging Face. Per the terms:

• A "Model Derivative" includes modifications and models trained to transfer Gemma's weights or patterns, but not its outputs.
• If you redistribute a derivative, you must pass on the use restrictions (Prohibited Use Policy), give recipients a copy of the terms, mark modified files, and include a notice file saying Gemma is provided under those terms.

Read the full terms before shipping a derivative.