llamaperf

Qwen3.8 27B AREX-2

on NVIDIA RTX 4090 · ExLlamaV3 · 8,192 ctx

NVIDIA hardware
Oct 9, 2026
Throughput
35.9 t/s gen
Quant
EXL3 4.00 bpw (EXL3)
KV cache
FP16
VRAM reported
24 GB

Use cases

agenticcodingtool-usevisionlong-contextmath

Summary

User reports AREX-2, a 27B dense multimodal agent model built on Qwen3.8-27B, at 35.94 tok/s decode on an RTX 4090 24 GB. Setup is ExLlamaV3 with an EXL3 4.00 bpw quant and FP16 cache, 8192-token runtime context, batch size 2, ~17 GiB artifact. The figure is a small local smoke benchmark, not a standardized result; a TabbyAPI smoke test on the same card saw about 49.4 tok/s on a very short request. Long-context quality and memory scaling at 32K/64K/262K were not validated.