How to fine-tune Falcon3 (LoRA, QLoRA)

See the Falcon3 hub for sizes. LoRA and QLoRA are the usual routes on a single GPU. Estimate cost with the training cost calculator or read the fine-tuning overview.

Which checkpoint

• The Base checkpoint is, in the card's words, a raw pretrained model that should be further fine-tuned for most use cases. Start here for full control of behavior.
• The Instruct checkpoint was post-trained on 1.2 million samples of STEM, conversation, code, safety and function-call data. Start here to adapt a model that already follows instructions.

The cards document no LoRA target modules; use your library's defaults for this architecture (28 decoder blocks with grouped query attention).

Chat template

Format Instruct training data with the tokenizer's apply_chat_template using system and user roles, so training matches inference. The card describes no reasoning toggle.

Languages and context

The cards list English, French, Spanish and Portuguese, and a 32K context window.

License limits on derivatives

The TII terms for Falcon 3 and newer models say:

• Include prominently in any public statement about a derivative: "[name of derivative] is built using artificial intelligence technology from the Technology Innovation Institute" (TII allows reasonable adjustments on request).
• Your use must comply with TII's Acceptable Use Policy, and distribution agreements must carry use-based restrictions that incorporate it.
• Give recipients a copy of the license, mark modified files, and retain existing notices.
• Copyright and patent licenses are royalty free.

Read the full license before shipping a derivative commercially.