llamaperf
Nov 20, 2026
Throughput
40.0 t/s gen · 1000.0 tokens/s prompt processing (prefill)Prompt processing is how fast the input is read, not how long you wait for the first word. How to measure time to first token.
Quant
IQ3_S (GGUF)
System RAM
64 GB
VRAM reported
12 GB

Summary

User compares two IQ3_S quant builds of Qwen3.8 Flash-Next on an RTX 4070 12GB with a Ryzen 5600X and 64GB DDR4, running Strata. The IQ3_S build decodes at 40 t/s with 1000 t/s prefill, which the user says is 20% slower than the Gyro-M build. The 125B-A6B model cannot fit in 12GB of VRAM, so weights ran partly from system RAM.