llamaperf

Qwen3.8 27B

on NVIDIA RTX 3090 · ThunderLlamaX · 100,352 ctx

Tone: positiveNVIDIA hardware
Oct 5, 2026
Throughput
75.8 t/s gen · 569.2 tokens/s prompt processing (prefill)Prompt processing measures input prefill speed; it does not tell you the time to first token.
Quant
IQ3_XXS (GGUF)
System RAM
16 GB
VRAM reported
24 GB

Use cases

codinglong-context

Summary

User reports Qwen3.8-27B dense at 75.81 tok/s decode on an RTX 3090 eGPU attached to a MacBook Air M2 over Thunderbolt 4. Setup is the ThunderLlamaX engine with IQ3_XXS GGUF weights, 100,352-token context, K=10 n-gram LOOKUP speculative decoding, and a custom DriverKit dext replacing the missing NVIDIA driver stack. Prefill is 569.2 tok/s at 2k, 510.1 at 8k, and 342.0 at 100k. On novel prose the EAGLE draft head gives 43.7 tok/s. A Qwen3.6-35B-A3B MoE model runs at 97.6-104.0 tok/s quote-class and 40.1 tok/s prose with 982.0 tok/s prefill at 2k. GSM8K is 95.0% dense and 93.0% MoE. A 100k context restores from cache in about 6.5 seconds.