Qwen3.8 27B AREX-2
on NVIDIA RTX 4090 · ExLlamaV3 · 8,192 ctx
Use cases
agenticcodingtool-usevisionlong-contextmath
Summary
User reports AREX-2, a 27B dense multimodal agent model built on Qwen3.8-27B, at 35.94 tok/s decode on an RTX 4090 24 GB.
Setup is ExLlamaV3 with an EXL3 4.00 bpw quant and FP16 cache, 8192-token runtime context, batch size 2, ~17 GiB artifact.
The figure is a small local smoke benchmark, not a standardized result; a TabbyAPI smoke test on the same card saw about 49.4 tok/s on a very short request. Long-context quality and memory scaling at 32K/64K/262K were not validated.