Qwen3.8 27B
on NVIDIA RTX 3090 · 64,000 ctx
Summary
User reports Qwen3.8-27B at 99 t/s single request and 1150 t/s batched on an RTX 3090.
Setup uses a hyper-optimized inference engine with an fp8 KV cache, int8 lm_head and embeddings, fp16 recurrent state, and int8 activations, at 64k context in fast mode.
Prefill reaches 1812 t/s at 1k context and 1000 t/s at 100k context.