Qwen3.8 27B Jev
NVIDIA RTX 4090 · TabbyAPI · 1,035 ctx
- generation:
- 53.7 tokens/s
- prompt processing (prefill):
- 885.0 tokens/s
- quant:
- 4.0bpw (EXL3)
- kv:
- FP16
User reports Jev_Qwen3.8-27B at 53.7 t/s decode and 885 t/s prefill on an RTX 4090. Setup is TabbyAPI with EXL3 4.0 bpw and FP16 KV cache, single GPU, max batch size 1, thinking disabled. Decode stayed near 54 t/s at short context and 48.3 t/s at 32K. A warmed TabbyAPI smoke test generated at 53.52 t/s, while a direct ExLlamaV3 probe measured 40.475 t/s. Observed GPU memory was approximately 16.4-16.7 GiB.