llamaperf

Qwen3.8 27B

on NVIDIA RTX 3090 · open-jet · 120,000 ctx

Tone: positiveNVIDIA hardware
Oct 16, 2026
Throughput
151.3 t/s gen
Quant
UD-Q4_K_XL (GGUF)
KV cache
D' (K 6-bit, V 5-bit, 8-bit ring)
VRAM reported
24 GB

Use cases

coding

Summary

User reports Qwen3.8-27B at 151.3 tok/s greedy decode on short prompts on one RTX 3090, using a fork of the open-jet engine with the model's own MTP head (7 drafts) and a custom D' KV cache (K 6-bit, V 5-bit, 8-bit ring for the newest ~2.5k tokens). Setup is the open-jet fork with UD-Q4_K_XL weights at 120k context, single stream, one request at a time. The same engine with q8 KV gives 150.2 tok/s; llama.cpp tuned with MTP 4 drafts and q8_0 KV gives 82.7 tok/s on the same card. Sampled thinking runs at 134.9 tok/s, decode at 38k context at 114.3 tok/s, and at 101.6k context at 105.4 tok/s. The full 262,144-token context fits on the card at 23.4 GB, reading 260,896 tokens in 334 s (781 tok/s) then decoding at 63 tok/s. Greedy token ids are identical at draft depths 0, 4 and 7, so output is unchanged with speculation off. The user notes the card is an ASUS Strix 3090 with a raised power limit (~470 W) and memory at 9,851 MHz, and that numbers will differ on other hardware.