llamaperf

Qwen3.8 177B Flash-Next Uncensored

on 4× NVIDIA P102-100 · llama.cpp · 32,768 ctx

Tone: mixedNVIDIA hardware
Oct 7, 2026
Throughput
10.9 t/s gen · 19.7 tokens/s prompt processing (prefill)Prompt processing measures input prefill speed; it does not tell you the time to first token.
Quant
IQ3_XXS (GGUF)

Summary

User compares mainline llama.cpp against ik_llama.cpp on a hybrid 4-GPU rig with a Qwen3.8 Flash-Next 177B Uncensored IQ3_XXS model at 32k context, 35 layers offloaded to GPU and 13 on the CPU. Mainline reaches 19.70 t/s prompt eval and 10.86 t/s generation with 2,490 CUDA graphs reused; ik_llama.cpp drops to 5.48 t/s prompt eval and 8.43 t/s generation with no CUDA graphs and context checkpoints during generation. User asks whether ik_llama.cpp's advantage applies only to CPU or single-GPU setups, and whether its threadpool struggles with pipelined layer splits across heterogeneous GPUs over PCIe.