Qwen3.8 27B
on NVIDIA RTX 3090 · open-jet · 120,000 ctx
Use cases
coding
Summary
User reports Qwen3.8-27B at 151.3 tok/s greedy decode on short prompts on one RTX 3090, using a fork of the open-jet engine with the model's own MTP head (7 drafts) and a custom D' KV cache (K 6-bit, V 5-bit, 8-bit ring for the newest ~2.5k tokens).
Setup is the open-jet fork with UD-Q4_K_XL weights at 120k context, single stream, one request at a time. The same engine with q8 KV gives 150.2 tok/s; llama.cpp tuned with MTP 4 drafts and q8_0 KV gives 82.7 tok/s on the same card.
Sampled thinking runs at 134.9 tok/s, decode at 38k context at 114.3 tok/s, and at 101.6k context at 105.4 tok/s. The full 262,144-token context fits on the card at 23.4 GB, reading 260,896 tokens in 334 s (781 tok/s) then decoding at 63 tok/s. Greedy token ids are identical at draft depths 0, 4 and 7, so output is unchanged with speculation off. The user notes the card is an ASUS Strix 3090 with a raised power limit (~470 W) and memory at 9,851 MHz, and that numbers will differ on other hardware.