Qwen3.5 4B
on NVIDIA RTX 2060 Super 8 GB; Intel Core i9-10900K host · llama.cpp · 8,192 ctx
Summary
User reports Qwen3.5 4B at 83.7 t/s on one NVIDIA RTX 2060 Super with 8 GB of VRAM and an Intel Core i9-10900K host.
Setup is llama.cpp server-cuda with Q4_K_M and a context of 8192 tokens, all layers forced onto the GPU.
The figure is the median of three requests after a short warm-up, with roughly 1% agreement among repeats; the source reports 3.3 GB of graphics memory in use.