Qwen3.5 9B

on NVIDIA RTX 4060 Laptop 8GB · llama.cpp

Oct 12, 2026

Throughput

37.7 t/s gen · 1828.8 tokens/s prompt processing (prefill)Prompt processing measures input prefill speed; it does not tell you the time to first token.

Quant

Q4_K_M (GGUF)

System RAM

24 GB

VRAM reported

8 GB

Summary

User reports Qwen3.5 9B at 37.7 t/s generation and 1828.8 t/s prompt processing on an NVIDIA RTX 4060 Laptop 8GB. Setup is llama.cpp with Q4_K_M and full GPU offload via -ngl 99, run through native llama-bench CUDA on a Lenovo LOQ 15IRX9 with 24GB system RAM. The figures come from separate synthetic tests, pp512 and tg128, not a single 512-token request followed by a 128-token completion; context allocation and KV precision are not given.