Granite Switch 4.1

1 report

Granite Switch 4.1 8B at 8-bit generates a median of 56.0 tokens per second on the M5 Max 128GB, the hardware it is most run on, from one plain run (community reports, as of 26 Oct 2026).

As of 26 Oct 2026, Granite Switch 4.1 8B at 8-bit on the hardware it is most run on, with the median of plain runs (one device, one request, no speculative decoding, the whole model in its memory):

M5 Max 128GB
56.0t/s · one run

Granite Switch 4.1 VRAM requirements by size and quant →

How does Granite Switch 4.1 run on your hardware?

Pick your GPU or Mac, then read the speeds people reported for Granite Switch 4.1 on it, or estimate memory fit and speed.

Buying hardware? Compare GPUs →

Free to use. No account needed. Memory estimates and community measurements are labelled separately.

Run Granite Switch 4.1 yourself? Add your numbers.

Your coding agent can run a speed test on your machine and send it here with a free account. Or paste a result you've already posted.

Benchmark with your agent Paste a result

Reported performance for Granite Switch 4.1

Filter this model’s reports by setup →

Thin page (1 of 3 reports needed for indexing). Add yours.

Granite Switch 4.1 8B

M5 Max 128GB · mlx-lm

generation:: 56.0 tokens/s
quant:: 8-bit (MLX)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.

tool-useagentic

User reports Granite Switch 4.1 8B at 56 t/s chat on an M5 Max 128GB, with the 30B at 17 t/s. Setup is mlx-lm with a custom granite_switch package, 8-bit MLX weights, greedy decoding, one request at a time, medians of 3 runs. The 4-bit builds reach 98 t/s (8B) and 32 t/s (30B). Decision steps take 0.62 s on the 8B and 2.3 s on the 30B at a ~1.2K-token prompt. Accuracy on 500 labelled claims is 0.814 (8B 8-bit), 0.844 (30B 8-bit) and 0.840 (30B 4-bit), against 0.812 and 0.846 for the bf16 conversions. The user picks the 8B 8-bit for routing and classification.

Get a weekly email of new Granite Switch 4.1 reports on any GPU.

Email me new reports

Reported configurations

Individual observations from this page, not expected speeds or a ranking. Select a model to open its report, with the source and offloading conditions. Reported context may be a limit rather than actual input length.

Model / hardware	Quant / engine	Reported context	Generation
Granite Switch 4.1 8B M5 Max 128GB	8-bit mlx-lm	Not reported	56.0 tokens/s

Frequently asked

What hardware do people run Granite Switch 4.1 on?

The hardware Granite Switch 4.1 8B at 8-bit is most run on: M5 Max 128GB (56.0 t/s median, one run). Medians of plain runs, as of 26 Oct 2026.