Qwen3.8 27B QUASAR
on 2× NVIDIA RTX 5070 Ti · capy-flash
Use cases
tool-usevision
Summary
User reports Qwen3.8-27B QUASAR-QAT NVFP4 decoding at approximately 130-168 tokens/s on 2x RTX 5070 Ti 16GB under WSL2.
Setup is capy-flash v0.6.0 built from source, with tensor parallelism across both GPUs and MTP enabled through the 27b profile; MTP acceptance was approximately 49-83%. Model weights are 17.0 GiB, weight loading took 12.1 seconds and total engine startup 29.7 seconds.
Qwen3.8-Flash-Next NVFP4 fails at startup with a cudaHostAlloc out-of-memory error on a host-mapped expert weight buffer of 70039033856 bytes (about 65.2 GiB); the user asks whether WSL2 can support such an allocation or whether the weights could be mapped in smaller chunks.