llamaperf

Qwen3.8 27B

on NVIDIA RTX 3090

Tone: positiveNVIDIA hardware
Oct 11, 2026
Throughput
1600.0 tokens/s prompt processing (prefill)Prompt processing measures input prefill speed; it does not tell you the time to first token.
Quant
Q4_K_M (GGUF)
KV cache
fp16

Use cases

coding

Summary

User reports Qwen3.8-27B at 140 tok/s on code generation on a single RTX 3090, using a custom CUDA megakernel that runs each speculative-decoding cycle in one launch. Setup is the megakernel with Q4_K_M weights and fp16 KV cache up to about 80k context, then q8. The same hardware with llama.cpp and MTP gives 73 tok/s on code, 56 tok/s on coding with a 4K-token file in context, and about 1,100 tok/s prefill. The megakernel is 1.4-1.9x faster than llama.cpp depending on task and context length, and is built for this model on 3090s only.