Local model performance on your hardware

generation:

187.0 tokens/s

quant:

FP8 (FP8)

User reports Metis-1.6 Logos decoding FP8 at 187 tok/s single-stream on an RTX 5070 Ti with the author's Dromos engine, against 118 tok/s for vLLM. Dromos speculates from context for about 1.3x on chat in direct mode; the model is a looped Mamba-2 plus latent-MoE design with 10.23B total and 1.18B active parameters and 131K context. Across 32 streams Dromos reaches 1,331 tok/s versus vLLM's 1,143. The post is a release write-up: the author flags that AIME, HMMT and other olympiad benchmarks are not yet in the decontamination manifest, and that thinking runs a median of about 25k tokens.