llamaperf

Flint 40B (2.5B active)

on NVIDIA RTX Pro 6000 Blackwell · llama.cpp · 65,536 ctx

Tone: positiveNVIDIA hardware
Oct 24, 2026
Throughput
244.0 t/s gen · 5230.0 tokens/s prompt processing (prefill)Prompt processing measures input prefill speed; it does not tell you the time to first token.
Quant
Q8_0 (GGUF)
VRAM reported
96 GB

Use cases

codingagentic

Summary

User benchmarks Flint 40B-A2.5B at 244 t/s single-request decode and 5,230 t/s prefill on one RTX PRO 6000 Blackwell 96 GB, comparing it with Ornith 1.5 35B-A3B and Nemotron 3.5 Lightning 30B-A3B in an agentic coding run. Setup is llama.cpp b11803 with Q8_0 GGUF, 64K context per slot and --parallel 3, three workers running in parallel behind Opus 5.5 as planner. Flint finished in 7:36 with 6 fixes and $0.41 of Opus spend, against Ornith's 9:04, 14 fixes and $0.49. Nemotron had the best decode and prefill on paper but generated about 5x more tokens and needed six fix waves. One run per model, variance not measured; Q8_0 on CUDA, so NVFP4/MLX numbers will differ.