How to fine-tune Granite 4.0 (LoRA, QLoRA)

This guide covers adapting the Granite 4.0 family with LoRA or QLoRA. Estimate cost with the training cost calculator and see fine-tuning for the overview.

Which checkpoint

• The -base checkpoints are pretrained decoder-only models. The base card says they "can serve as baseline to create specialized models for specific application scenarios" and support Fill-in-the-Middle code completion.
• The instruct checkpoints are the right start for chat, tool use and RAG-style behaviour, since they already carry the Granite chat template.
• Sizes: Micro Dense (3B), H Micro Dense (3B), H Tiny MoE (7B), H Small MoE (32B, 9B active).

Chat template

Train with the model's own template, which wraps turns in <|start_of_role|> and <|end_of_role|> tokens. Tool calls are JSON inside <tool_call> tags following the OpenAI function schema, so format tool-use data the same way. The instruct card notes a default system prompt added in October 2025.

Architecture notes

H models mix 36 Mamba2 layers with 4 attention layers, and H Small and H Tiny use mixture-of-experts. The cards do not document recommended LoRA target modules, so choose them from the module names in the checkpoint and verify your trainer supports the Mamba2 layers. The cards give no guidance on MoE routing during fine-tuning.

License

The checkpoints are listed as Apache 2.0, which allows commercial use and derivatives. Read the license file shipped with the repo for attribution terms.

Languages

The cards say users may fine-tune for languages beyond the 12 supported ones.