llamaperf

LFM2.5 8B (1B active)

on NVIDIA RTX 5090 · llama.cpp

NVIDIA hardware
Oct 11, 2026
Throughput
563.1 t/s gen · 30680.7 tokens/s prompt processing (prefill)Prompt processing measures input prefill speed; it does not tell you the time to first token.
Quant
Q8_0 (GGUF)
System RAM
64 GB
VRAM reported
32 GB

Summary

User reports LFM2.5-8B-A1B at 563.12 t/s generation and 30,680.66 t/s prompt processing on an RTX 5090. Setup is llama.cpp llama-bench (build b9653) with Q8_0 weights, all layers on GPU (ngl=99), single request, no speculative decoding. The dataset card also lists llama-bench rows for gpt-oss-20b, Nemotron-3-Nano-30B-A3B, Qwen3.6-35B-A3B, Qwen3-Coder-Next, Gemma 4 12B, Qwen3.6-27B, Gemma 4 31B, and Ling-3.0-flash, plus rows for Qwen3.6-27B-MTP, Qwen3.6-27B Q6_K, gpt-oss-120B, Mistral Small 4 119B, and several quant ladders; some runs used vLLM and SGLang.