Qwen3.8 27B
on NVIDIA RTX 3090 · ThunderLlamaX · 100,352 ctx
Use cases
codinglong-context
Summary
User reports Qwen3.8-27B dense at 75.81 tok/s decode on an RTX 3090 eGPU attached to a MacBook Air M2 over Thunderbolt 4.
Setup is the ThunderLlamaX engine with IQ3_XXS GGUF weights, 100,352-token context, K=10 n-gram LOOKUP speculative decoding, and a custom DriverKit dext replacing the missing NVIDIA driver stack.
Prefill is 569.2 tok/s at 2k, 510.1 at 8k, and 342.0 at 100k. On novel prose the EAGLE draft head gives 43.7 tok/s. A Qwen3.6-35B-A3B MoE model runs at 97.6-104.0 tok/s quote-class and 40.1 tok/s prose with 982.0 tok/s prefill at 2k. GSM8K is 95.0% dense and 93.0% MoE. A 100k context restores from cache in about 6.5 seconds.