Qwen3.8 27B
on 2× NVIDIA RTX 5060 Ti 16GB · vLLM · 122,880 ctx
Use cases
codingmathtool-uselong-context
Summary
User compares NVFP4 on vLLM against Q6_K on llama.cpp, both on 2x RTX 5060 Ti.
NVFP4 reaches 952.5 t/s prefill and 67.3 t/s decode with 0.696 MTP acceptance. Q6_K reaches 597.9 t/s prefill and 38.6 t/s decode.
On quality, NVFP4 scores 65/68 correct, 64/68 exact format and 61/68 both, while Q6_K scores 66/68, 68/68 and 66/68. Both score 10/10 on long-context retrieval. NVFP4 is the default endpoint, with Q6_K kept for stricter output.