Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports Qwen3.8-27B at about 68 t/s generation on dual RTX 5060 Ti.
Setup is speculative decoding with a Q8 KV cache at 100k context. Vision is enabled but not used.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
creative-writing
User reports Qwen3.8 27B generating a GTA-style game, with prompt processing of ~850-890 t/s for 25k tokens and generation typically 50-55 t/s, peaking at 197 t/s with ngram speculative decoding.
Setup is llama.cpp with ngram and MTP draft, a Q8 KV cache, and 128k context.
User is enthusiastic about the results.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports Qwen3.8-27B on an RTX 5060 Ti 16GB with llama.cpp, comparing the jpetrina IQ4_XS-pure and Unsloth UD-IQ4_XS quants.
Short-context generation runs at roughly 45-47 tok/s, dropping to about 31 tok/s after 55K prefill. The Unsloth quant has better fidelity (KLD 0.018 vs 0.0236), while jpetrina wins the multi-turn agent eval (40/80 vs 38/80).
Vision works but needs a separate profile. A Q8 baseline was used as a quality control.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports a speed gap between LM Studio at 30-40 tok/s and raw llama-server at 50-55 tok/s on the same model and settings.
The model is a Q6_K quant.
User also asks about reasoning effort dropdown differences between lmstudio-community and Unsloth GGUFs.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports Qwen3.8-27B q8 GGUF at 20 t/s tg and 1000 pp on a Threadripper rig with 4x RTX 5060 Ti 16GB, VRAM only.
User also reports DeepSeek V4 Flash 0731 GGUF at 11 t/s tg and 200 pp with RAM offload.
User asks about vLLM tensor parallelism, NVFP4, a CPU upgrade, and other models.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports over 200k context on 16 GB of VRAM with Qwen 3.8 27B at the IQ3_XXS quant.
Setup is a laptop with Thunderbolt 4 and an Aorus 5060 Ti AI Box eGPU on Windows 11, with the KV cache quantized to q5_1.
The user previously ran UD-Q3_K_XL at 140k context. Prompt processing dropped from 700-800 tk/s to 400 tk/s.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports 46 t/s generation and 700 t/s prefill with tensor parallelism enabled, against 28 t/s generation and 1000+ t/s prefill without it.
Setup uses MTP with max draft tokens 6 and probability 0.88.
Draft acceptance is 0.93757 with a mean length of 5.10.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports Qwen3.8-27B-NVFP4-MTP-GGUF at 210k context on 3x RTX 5060 Ti 16GB in a Dell 5820 workstation with 128GB DDR4 RAM, with prompt processing at 1300-700 t/s and generation at 29-71 t/s, mostly 38-44.
Setup is llama.cpp with a Q8 KV cache and vision enabled.
The user added a third GPU but reports cooling and stability issues, and is considering a second workstation with subagents for Mixture of Agents. The user also tested Ornith 1.5 35B (MXFP) and Gemma 4 26B NVFP on 16GB VRAM. Use cases are web hosting, office documents, and prompt generation for image and video models.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
coding
User reports 608 t/s cold prefill and 52.2 t/s decode on 4x RTX 5060 Ti 16GB at 256K context.
Setup is a Vast AI instance with the Q8 quant, an FP16 KV cache, and MTP enabled.
User considers this excellent for $2K hardware.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports 50-54 t/s with a custom CUDA/C++ engine, a 50% improvement over llama.cpp at 33-34 t/s.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports 19 t/s peak and 14 t/s during long generation at 65k context with the KV cache offloaded to RAM via -nkvo, which allows the whole model on GPU with an f16 KV cache.
Speed remained similar at 128k context with 63 layers on GPU.
Quantizing the KV cache to RAM did not improve performance.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports Qwen3.6 27B IQ4_XS at 22 t/s generation and 760 t/s prompt processing on an RTX 5060 Ti 16GB.
Setup is llama.cpp with TheTom's TurboQuant fork, a 75k context window, and a turbo4/turbo2 KV cache.
The user also tested BF16, Q8_0, Q6_K, Q5_K_XL, Q4_K_XL, IQ4_XS, IQ3_XXS, Q3_K_XL, Q3_K_M and Q2_K_XL on an L40S or RTX 5060 Ti, comparing quality with a chess board SVG generation task, and recommends IQ4_XS as the minimum.