Use cases
coding
Summary
User reports Qwen3.8-27B at 140 tok/s on code generation on a single RTX 3090, using a custom CUDA megakernel that runs each speculative-decoding cycle in one launch.
Setup is the megakernel with Q4_K_M weights and fp16 KV cache up to about 80k context, then q8. The same hardware with llama.cpp and MTP gives 73 tok/s on code, 56 tok/s on coding with a 4K-token file in context, and about 1,100 tok/s prefill.
The megakernel is 1.4-1.9x faster than llama.cpp depending on task and context length, and is built for this model on 3090s only.