How to fine-tune Qwen3.5 (LoRA, QLoRA)

This guide covers adapting models from the Qwen3.5 family. VRAM and GPU counts appear below. See also LoRA, QLoRA, the training cost calculator and fine-tuning.

Which checkpoint to start from

• The main repositories (for example Qwen3.5-35B-A3B) are post-trained.
• Base checkpoints are published for several sizes: the Qwen organization lists Qwen3.5-0.8B-Base, 2B-Base, 4B-Base, 9B-Base and 35B-A3B-Base.
• Start from Base for domain adaptation or your own instruction tuning. Start from the post-trained model to keep its chat and reasoning behavior.

Chat template and thinking

The post-trained models think by default (<think> blocks) and enable_thinking can be set to false through chat_template_kwargs. Train with the shipped template and keep the thinking format consistent across your dataset. If your data has no reasoning traces, train in the non-thinking format.

Architecture notes

The 35B-A3B model is a Mixture-of-Experts model, and the family is a unified vision-language design trained with early fusion. The card does not document LoRA target modules or adapter settings, so inspect the checkpoint's module names. If you only want text behavior, keep the vision components frozen.

For libraries, the card points to Transformers main at release time, so use a recent version.

License

Qwen3.5-35B-A3B is released under Apache 2.0, which permits commercial use and derivative works under its standard terms. Confirm the license field on any other size you use.