llamaperf

Qwen3.8 27B QUASAR

on 2× NVIDIA RTX 5070 Ti · capy-flash

Tone: mixedNVIDIA hardware
Oct 19, 2026
Throughput
130-168 t/s gen
Quant
NVFP4 (NVFP4)
System RAM
128 GB
VRAM reported
16 GB

Use cases

tool-usevision

Summary

User reports Qwen3.8-27B QUASAR-QAT NVFP4 decoding at approximately 130-168 tokens/s on 2x RTX 5070 Ti 16GB under WSL2. Setup is capy-flash v0.6.0 built from source, with tensor parallelism across both GPUs and MTP enabled through the 27b profile; MTP acceptance was approximately 49-83%. Model weights are 17.0 GiB, weight loading took 12.1 seconds and total engine startup 29.7 seconds. Qwen3.8-Flash-Next NVFP4 fails at startup with a cudaHostAlloc out-of-memory error on a host-mapped expert weight buffer of 70039033856 bytes (about 65.2 GiB); the user asks whether WSL2 can support such an allocation or whether the weights could be mapped in smaller chunks.