Tencent-HY3 VRAM requirements
Memory needed to run Tencent-HY3 at every published size and quant, with an 8k context. Each figure is weights plus KV cache plus runtime buffers, from the same memory model as the VRAM calculator.
Total memory by size and quant
GB at an 8k context. Bold is the Q4_K_M column, the usual starting point.
| Size | Q8_0 | Q6_K | Q5_K_M | Q4_K_M | Q3_K_M |
|---|---|---|---|---|---|
| 295B · 21B active | 299 | 244 | 207 | 170 | 133 |
KV cache by context length
GB the context alone adds on top of the weights. Add this to a weights figure to size a longer session.
| Size | Weights at Q4_K_M | 4k ctx | 16k ctx | 32k ctx | 128k ctx |
|---|---|---|---|---|---|
| 295B · 21B active | 166 | 1.1 | 4.3 | 8.6 | 34 |
What each size fits on at Q4_K_M
Hardware whose memory holds the model with headroom, smallest pool first. Fits means under 85% of the pool at an 8k context.
- 295B · 21B activeneeds about 170 GB
Smallest: AMD Threadripper 256GB (256 GB), M3 Ultra 256GB (256 GB), M5 Ultra 256GB (256 GB), M3 Ultra 512GB (512 GB), and 1 more.
Frequently asked
How much VRAM does Tencent-HY3 need?
About 170 GB at Q4_K_M with an 8k context for the 295B · 21B active model, including the KV cache and runtime buffers. Q8 needs more and Q3 less; the table above lists every rung.
Can I run Tencent-HY3 on a 16GB GPU?
Not comfortably. Even the 295B · 21B active model needs about 170 GB at Q4_K_M with an 8k context, which is more than a 16 GB card holds with headroom. A lower quant or CPU offloading can still get it running, at a cost in quality or speed.
Can I run Tencent-HY3 on a 24GB GPU such as an RTX 3090 or 4090?
Not comfortably. Even the 295B · 21B active model needs about 170 GB at Q4_K_M with an 8k context, which is more than a 24 GB card holds with headroom.
How are these numbers calculated?
Weights at the quant's bits per weight, plus a KV cache sized from the model's attention design for the chosen context, plus compute buffers and a fixed runtime allowance. It is the same memory model the VRAM calculator uses. A family whose attention profile is not on file is sized as plain grouped-query attention, which errs towards needing more memory.