Qwen3-Coder 30B (3B active)
NVIDIA RTX 5090 Laptop 24GB · llama.cpp · 65,536 ctx
- generation:
- 107.0 tokens/s
- prompt processing (prefill):
- 3706.5 tokens/s
- quant:
- Q4_K_M (GGUF)
- kv:
- Q8_0
User reports Qwen3-Coder-30B-A3B-Instruct at 107.0 tok/s generation and 3,706.5 tok/s prefill on an RTX 5090 Laptop 24GB in an MSI Raider A18. Setup is llama.cpp with CUDA, Q4_K_M GGUF (17.28 GiB), Q8_0 KV cache, FlashAttention on, 65,536-token context, all layers on the GPU, one slot. The figures come from a repository-analysis workload with an 8,638-token prompt and 371 generated tokens; the user notes the screenshot captures one strong interval, not an average across the audit.