Qwen3.8 177B Flash-Next Uncensored
on 4× NVIDIA P102-100 · llama.cpp · 32,768 ctx
Summary
User compares mainline llama.cpp against ik_llama.cpp on a hybrid 4-GPU rig with a Qwen3.8 Flash-Next 177B Uncensored IQ3_XXS model at 32k context, 35 layers offloaded to GPU and 13 on the CPU.
Mainline reaches 19.70 t/s prompt eval and 10.86 t/s generation with 2,490 CUDA graphs reused; ik_llama.cpp drops to 5.48 t/s prompt eval and 8.43 t/s generation with no CUDA graphs and context checkpoints during generation.
User asks whether ik_llama.cpp's advantage applies only to CPU or single-GPU setups, and whether its threadpool struggles with pipelined layer splits across heterogeneous GPUs over PCIe.