IBM Granite models, generation by generation

IBM's Apache 2.0 Granite LLMs from Granite 3.0 to 4.2: sizes, dense vs hybrid Mamba MoE, context lengths, plus the code, Docling and speech side lines.

What the line is

Granite is IBM's family of open models, published on Hugging Face under the ibm-granite organization. Every card checked for this page lists the Apache 2.0 license, which is the main reason developers pick it for commercial work: no custom terms to review. The language models are instruct-tuned decoder-only Transformers, and the 4.0 generation added a hybrid design that mixes attention with Mamba2 layers.

The main line runs Granite 3 (October 2024) through Granite 4.2 (August 2026). Around it are code, document-conversion and speech models, each under its own hub.

Generations in order

Granite 3 (October 2024 to April 2025)

Granite 3 covers four point releases, all Apache 2.0:

  • 3.0 (October 21, 2024): 2B and 8B dense models plus 1B and 3B mixture-of-experts models, per the card's size table. The 8B is a dense decoder with grouped-query attention and RoPE. The 1B-A400M is a sparse MoE with fine-grained experts. The 8B config sets a 4,096-token maximum position.
  • 3.1 (December 18, 2024): the 8B instruct card calls it a long-context model, and its config sets 131,072 positions.
  • 3.2 (February 26, 2025): the 8B instruct card includes a step-by-step reasoning example in its usage section.
  • 3.3 (April 16, 2025): the 8B instruct card describes an 8-billion-parameter model with a 128K context window, fine-tuned for better reasoning and instruction following.

Granite 4.0 (October 2025)

Granite 4 begins with the 4.0 release on October 2, 2025. The H models use a hybrid decoder: grouped-query attention, Mamba2 layers, MoE blocks with shared experts, SwiGLU and RMSNorm. The cards give these sizes:

  • Granite-4.0-H-Small: about 32B total parameters, mixture-of-experts.
  • Granite-4.0-H-Tiny: about 7B total parameters, mixture-of-experts.
  • Granite-4.0-Micro: about 3B, a conventional dense Transformer.
  • Granite-4.0-350M: dense, released October 28, 2025.

The larger 4.0 models list a 128K context, and the configs set 131,072 positions.

Granite 4.1 (April 2026)

Granite 4.1 was released on April 29, 2026 in 3B, 8B and 30B sizes. The cards describe each as a decoder-only dense Transformer with grouped-query attention and RoPE, so this generation went back to plain dense models after the hybrid 4.0 release. The configs set 131,072 positions. The 4.1 models are in the Granite 4 hub, along with vision, speech and guardian models built on this generation.

Granite 4.2 (August 2026)

Granite 4.2 was released on August 25, 2026, also in 3B, 8B and 30B dense sizes. Its main change is built-in reasoning: the cards describe <think> chain-of-thought with full, non-thinking and low-effort modes, and reasoning-aware tool calling. Context is 128K natively, with a stated extension to 512K. The 8B card names Granite-4.1-8B-Base as its base model. License is Apache 2.0. It is the newest generation listed here, and it is also in the Granite 4 hub.

Side lines

  • Granite Code: code models. The Granite-3B-Code-Base-2K card describes a decoder-only code model trained from scratch in two phases, the first on 4 trillion tokens from 116 programming languages, released May 6, 2024. The 2K in the name is its context window.
  • Granite Docling: granite-docling-258m, released September 17, 2025, for document conversion. It builds on the Idefics3 architecture with a SigLIP2 vision encoder and a 165M-parameter Granite language model.
  • Granite Speech: speech models. granite-speech-3.3-8b (revision 3.3.2 dated June 19, 2025) pairs a speech encoder with granite-3.3-8b-instruct. Granite-speech-5.0-470m-turboctc (August 25, 2026) is a different design: 16 conformer blocks trained with CTC.

Which generation to use today

Picks come from sizes, context and license, not from benchmarks.

  • Default for chat, tools and agents: Granite 4.2 at 8B, for its reasoning mode, 128K native context and Apache 2.0 license. Move to the 3B for small GPUs or the 30B for more capacity.
  • Non-reasoning instruct model: Granite 4.1 in the same three sizes.
  • Hybrid attention and Mamba2 design: the 4.0 H models, where the cards describe attention combined with Mamba2 layers. Check that your inference engine supports the hybrid architecture first.
  • Tiny model on constrained hardware: Granite-4.0-350M.
  • Older stacks pinned to Granite 3.x: 3.3 is the one with a stated 128K window; the 3.0 config limits positions to 4,096.

You can run Granite models on Aquanode GPUs.

Granite generations

Every Granite generation we track, oldest first. Each hub lists all of its models with VRAM at native, FP8 and INT4 precision.

HubModelsSizesSmallest native VRAMCheapest live fit for itEst. $/hr
Granite 391.3B to 8.2B3.0 GBRTX 3070$0.088/hr
Granite 422352M to 32.2B0.8 GBRTX 3070$0.088/hr

Smallest native VRAM is the lowest requirement among the publisher's own checkpoints in the hub, at the precision they are published in; see the methodology.

Other Granite lines

Side lines from IBM: specialised or companion models outside the main Granite generations.

HubModelsSizesSmallest native VRAMCheapest live fit for itEst. $/hr
Granite Code13.5B7.8 GBRTX 3070$0.088/hr
Granite Docling1258M0.6 GBRTX 3070$0.088/hr
Granite Speech7473M to 8.6B1.1 GBRTX 3070$0.088/hr

Run and fine-tune Granite

Per-generation guides: engine support, chat template and context flags, then fine-tune memory for LoRA and QLoRA.

Granite compared with other models

Side-by-side VRAM, context length and license for models of a similar size.

Best models by task

Granite models appear on these ranked task pages.

Sources

Facts in the text above were read from these pages and are the publisher's own statements, not benchmarks run by Aquanode. Last reviewed 2026-10-07.

Related on Aquanode: GPU requirements for granite-4.0-h-small, All Granite Code models and their GPU requirements, GPU requirements for granite-3b-code-base-2k, and All Granite Docling models and their GPU requirements.

Start ComfyUI on a rented GPU.

© 2026 Aquanode. All rights reserved.

All trademarks, logos and brand names are the property of their respective owners.