132.0 t/s gen · 2000.0 tokens/s prompt processing (prefill)Prompt processing measures input prefill speed; it does not tell you the time to first token.
VRAM reported
24 GB
Summary
User reports a custom kernel achieving fp32 quality with 0.99997 similarity at int8.
Setup includes optimized prefill and decode speeds.