Sep 7, 2026
Summary
User reports 80-120 t/s generation and 12k t/s prefill for a single request on 4x R9700.
Setup is an optimized vLLM image with tcclaviger's MXFP4-FP8 quant, tensor-parallel-size 4, fp8 KV cache, and speculative decoding with MTP.