llamaperf
Submit

Qwen3.5 VRAM requirements

Memory needed to run Qwen3.5 at every published size and quant, with an 8k context. Each figure is weights plus KV cache plus runtime buffers, from the same memory model as the VRAM calculator.

Total memory by size and quant

GB at an 8k context. Bold is the Q4_K_M column, the usual starting point.

SizeQ8_0Q6_KQ5_K_MQ4_K_MQ3_K_M
0.8B2.72.52.42.32.2
2B4.03.63.43.12.9
4B6.15.34.84.33.8
9B119.48.37.16.0
13B1513119.47.8
27B2924211714
35B · 3B active3730262217
122B · 10B active125102867156
397B · 17B active399325275225176

KV cache by context length

GB the context alone adds on top of the weights. Add this to a weights figure to size a longer session.

SizeWeights at Q4_K_M4k ctx16k ctx32k ctx128k ctx
0.8B0.50.10.30.52.1
2B1.10.10.50.93.8
4B2.30.20.61.24.8
9B5.10.20.61.24.8
13B7.30.20.71.35.4
27B150.31.12.18.6
35B · 3B active200.10.30.72.7
122B · 10B active690.41.53.012
397B · 17B active2230.10.51.04.0

What each size fits on at Q4_K_M

Hardware whose memory holds the model with headroom, smallest pool first. Fits means under 85% of the pool at an 8k context. A Mac's pool is what macOS lets the GPU use - two thirds of its memory up to 32 GB, three quarters from 36 GB - not the whole of it.

Frequently asked

How much VRAM does Qwen3.5 need?

From about 2.3 GB for the 0.8B model to about 225 GB for the 397B · 17B active model, at Q4_K_M with an 8k context and including the KV cache and runtime buffers. Q8 needs more and Q3 less; the table above lists every size and rung.

Can I run Qwen3.5 on a 16GB GPU?

Yes. The 13B model needs about 9.4 GB at Q4_K_M with an 8k context, which fits a 16 GB card with headroom. Larger sizes need a smaller quant or a bigger card.

Can I run Qwen3.5 on a 24GB GPU such as an RTX 3090 or 4090?

Yes. The 27B model needs about 17 GB at Q4_K_M with an 8k context, which fits a 24 GB card with headroom. Larger sizes need a smaller quant or more memory.

How are these numbers calculated?

Weights at the quant's bits per weight, plus a KV cache sized from the model's attention design for the chosen context, plus compute buffers and a fixed runtime allowance. It is the same memory model the VRAM calculator uses. A family whose attention profile is not on file is sized as plain grouped-query attention, which errs towards needing more memory.