Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
tool-useagentic
User reports Qwen 3.8 27B at 36.5 t/s on an RTX 4070 Ti Super 16 GB with 64 GB system RAM.
Setup is Ollama on Windows with UD-Q3_K_XL and an 8-bit KV cache at 32K context.
The user says the 8-bit cache alone added about 26% generation speed and Q3+Q8 was about 78% faster than IQ4+Q8; other rows were Q4_K_M at 11.3 t/s, IQ4_XS at 16.3 t/s and IQ4_XS with 8-bit cache at 20.5 t/s. These are short tests on two warmed deterministic prompts, not an accuracy benchmark, and harder-task quality is untested.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
coding
User compares Swift-1.5-Qwen3.8-27B at around 13 tokens/sec against Strata on an RTX 4070 12 GB plus RTX 3060 12 GB with 32GB DDR4, running the same instruction runsheet.
Qwen3.8-27B runs Q4_K_M; Strata runs IQ3_XXS with experts streamed from an experts.bin file on an M.2 SSD, context varied from 96k to 160k, KV cache tried as Int8, q8v4 and q4_0, kv-resident 20480 to 32768.
Strata reached around 17 tokens/sec, 31-36 t/s for plain text generation and 20-25 t/s for logic/coding, with 5700 to 6800 experts in VRAM and system RAM maxed. Lowering kv-resident raised experts in VRAM but ran 3-4 t/s slower. User asks where the bottleneck is and which low-RAM flags to use.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports a GPU unlock module raising prefill on Qwen3.8 Flash-Next under Strata on 2 RTX 3090s, from 3205 to 3536 tok/s at 90K context, a 10% gain.
The module, gpu-unlock by Zagitta, sets FMLA32 to full before the NVIDIA driver loads; the user measured 4K and 10K prefill gains of 12% each.
BF16 tensor math rose from 82 to 155 TFLOPS and TF32 from 41 to 79 TFLOPS, but outputs were unchanged byte for byte over 1996 teacher-forced positions. The user asks whether it works on 40-series cards.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User asks whether any engines are optimized for consumer AMD hardware, noting most specialized engines target Nvidia.
They report running Swift 1.5 GSQ RCO at about 40-45 TPS on an RX 7800 XT, with no engine named.
The figure is given only as a range, and the user says they are happy with the setup and wonders how much more can be squeezed out of the hardware.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
agenticcoding
User benchmarks Swift-1.5-Qwen3.8-27B-AWQ-INT4 on two RTX 3080 20GB cards with the new GPU performance unlock, reporting +20.8% prefill at 8K (967 to 1168 tok/s) and no decode change.
Setup is vLLM 0.32.0 with tensor parallel 2, FP8 KV cache, MTP enabled with 3 speculative tokens, 240W power limit per card.
Prefill gains grow with prompt length, up to +28.6% at 261,900 tokens (562 to 723 tok/s). Decode is reported as steps/s (about 16), which the user says works out to roughly 45 tok/s in real agentic coding with MTP acceptance around 2. Both GPUs thermally throttled to 1750MHz.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports Qwen3.8-Flash-Next at about 40 t/s decode and 1000 t/s prefill on an RTX 3070.
Setup is Strata with an IQ3_S GGUF, int8 KV cache, 262144 max context, spec set to 4, and a q8 n-gram table instead of the bundled iq4_nl one. The model is 125B-A6B with experts partly offloaded, on a Ryzen 5 5600 with 64GB of RAM.
A vanilla config at the same context gave about 500 t/s prefill and 2-3 t/s less decode. The GPU unlock gave maybe 50-100 t/s extra prefill. The model runs from a SATA SSD, which the user thinks may hamper performance.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports Qwen3.8-27B at 140 tok/s on an M5 Max MacBook Pro with 128GB, the fastest run of their open source Apple Silicon engine kapitsa.
Setup is kapitsa with a 4bit model and kflash speculative decoding, where the 27B verifies a tree of drafter guesses in one pass rather than a single chain.
The user is asking Mac Studio owners to run the same model for comparison.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
coding
User benchmarks Qwen3.8-Flash-Next at 141.0 t/s generation and 3906.4 t/s prompt processing on an M5 Ultra 256GB.
Setup is oMLX with a Q6_K_XL GGUF at a 1024-token prompt, single request, no speculation.
Decode ranges from 67.6 to 197.3 t/s across the 1024 to 200000-token prompt sweep. Continuous batching reaches 316.4 t/s aggregate at 4 concurrent requests. User says opencode runs fast and responsively but code quality still has room for improvement.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
coding
User reports Qwen3.8 Flash-Next at 201 tok/s on 4x V100 32GB PCIe cards with no NVLink, for a single request of English code.
Setup is ReVolta 0.1.2, a vLLM fork, with an AWQ quant, 512K context, FP8 KV cache with CPU offload, TP4 plus DCP2, and DFlash2 speculative decoding drafting 8-token blocks from incoai's DFlash2 checkpoint.
The user says this may be the first DFlash run on Volta (SM70); the table also lists a 163 figure beside the 201 tok/s row.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
vision
User reports Qwen3.8 27B Vision at 3.82 t/s on a four-GPU mixed-vendor node (RX 580 8GB, RX 570 8GB, GTX 1660 Super 6GB, GTX 1070 8GB) with an i7-6700K.
Setup is Slipstream, a custom Nuitka-compiled Python engine, using a mixed-vendor Vulkan backend with Flash Attention across PCIe x1 and x2 lanes.
The figure is from concurrent serving of 27B, 9B and 2B models, handling OCR and vision parsing.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
codingagentic
User reports Qwen3.8-27B at 69.63 t/s generation on an RX 7900 XTX 24GB, with prompt processing at 383.02 t/s.
Setup is llama.cpp 2.61.0 (Vulkan/RADV) via LM Studio with Q4_K_M and q8_0 KV cache, 184320-token context, flash attention on, all layers on GPU and about 23.3 of 24 GiB VRAM in use.
Speculative decoding was on (MTP draft plus ngram, up to 2 draft tokens), so decode is inflated versus a plain run; draft acceptance was 0.558-0.68. A second 512-token run reached 64.09 t/s decode and 239.97 t/s prompt. One request at a time, nothing offloaded to system RAM.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports Qwen3.8-27B at around 7-9 t/s on an RX 7900 XT 20GB, sometimes lower, and asks why it is so slow compared with a desktop PC.
Setup is llama.cpp (llama-cli) with a Q4_K_M GGUF at 2048 context, -ngl 99, on a dual Xeon E5-2630 v3 server with 192GB DDR4 1866.
The speed is given only as a range, so no single figure is recorded.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports Qwen3.8-Flash-Next at about 10 tok/s generation and about 200 tok/s prefill on a Mac Mini M6 with 32 GB.
Setup is Strata with a Q2_0 quant at 64K context.
User says quality seems on par with 4-bit Qwen3.8-27B and that the fans almost never turn on with Strata, unlike the 27B run with Splash.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports Qwen3.8 27B at 37.09 t/s generation and 877.84 t/s prompt processing on an AMD Radeon RX 7900 XTX under llama.cpp's Vulkan backend.
Setup is llama.cpp (build 10438) with a UD-Q4_K_XL GGUF of 16.68 GiB, run with GGML_VK_DISABLE_HOST_VISIBLE_VIDMEM=1 because Resizable BAR is disabled in the BIOS; without that variable token generation is slow.
At a 4096-token context depth the same run gives 36.33 t/s generation and 812.34 t/s prompt processing.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports the Swift-1.5 fine-tune of Qwen3.8-27B at 36.5 tok/s decode on an RX 7900 XT 20GB, with MTP speculative decoding (acceptance τ≈1.94).
Setup is the hipfire runtime (v0.4.2.1+) with MQ4 weights and ROCm 7.1; the model fits in ~17.5 GB VRAM at 8K context, and 49K context with MTP is the recommended daily setting.
Prefill holds ~1,100 tok/s at 16K, and decode is 52–59 tok/s at 16–49K context. Offloading layers to system RAM past that drops decode to ~16–31 tok/s. The user claims ~2.2x faster end-to-end than Q4_K_S GGUF on llama.cpp, and notes the .mq4 files only load in hipfire and were tested only on gfx1100.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
codingagentic
User reports Qwen3.8-27B at 78.53 t/s generation and 523.33 t/s prompt processing on a Radeon RX 7900 XTX 24GB.
Setup is llama.cpp 2.61.0 (Vulkan backend, LM Studio) with Q4_K_M GGUF and q8_0 KV cache at 106496 context, whole model on the single GPU (16.8GB of 24GB VRAM). Speculative decoding was on (MTP + ngram, spec-draft-n-max 2), so the generation figure is an effective speed with draft acceptance of 78/98.
Four runs, one request at a time: generation ranged 61.68 to 78.53 t/s across runs. Runs 3 and 4 repeated a cached prompt (588 of 592 tokens), so their prompt speeds are startup time, not prefill. The user runs this model as a daily driver for coding and agent work.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
agentic
User reports Qwen3.8-Flash-Next at 150 tok/s decode and about 4,400 tok/s prompt on a single RTX 5080 16 GB.
Setup is Strata v0.2.1 with IQ3_XXS GGUF, int8 KV cache and 64K context, MTP speculation on, 15 CPU expert workers and an expert cache of 5,310 experts (8.61 GiB VRAM).
Decode median 150.0 tok/s over 9 runs (126.7-170.2); fresh prompt runs gave 3,755 tok/s at 4K, 4,396 at 16K and 4,412 at 32K. The card was tuned with LACT (memory +6000, core capped 2850 MHz, 360 W), so stock cards may be slower.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports 47 t/s generation.
Source evidence (verbatim):
Source: https://computingforgeeks.com/run-qwen3-8-27b-locally/
# Run Qwen3.8-27B Locally: VRAM and Speed Tested
Hardware: RTX 4090 24 GB, Ubuntu 24.04, llama.cpp build b11200 (CUDA 12.8), Ollama 0.34.4. Model Qwen3.8-27B unsloth UD-Q4_K_XL, flash attention on, -ngl 99. Without MTP (plain run), 8K context, f16 KV: 47.0 tok/s generation, 17.0 GiB VRAM. 64K f16: 47.0 tok/s. 131K q8_0: 46.6 tok/s. 262K q4_0: 46.5 tok/s, 22.2 GiB. llama-bench generation without MTP: Q4_K_XL 47.9, Q5_K_XL 41.5, Q6_K 39.6 tok/s. llama-bench prompt processing: 2,600 to 3,000 tok/s. A 196,971-token prompt: prefill 121.5 s (1,621 tok/s), generation 26.6 tok/s. With MTP speculative decoding (draft-mtp, mtp-Qwen3.8-27B-Q4_0.gguf) at 8K context: Q4_K_XL 109.4 tok/s (runs 102.8, 109.4, 113.2), Q5_K_XL 95.8 tok/s, acceptance 69%.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
coding
User reports Qwen3.8-27B at 190 tok/s on two RTX 3090s power-limited to 280 W over plain PCIe, at 0 context.
Setup is the Geezuz engine, a dual-GPU fork of L-Forster's open-jet megakernel, with Q4_K_M GGUF weights and 8-bit KV cache; the model drafts with its own MTP head, verifying 5 drafted tokens per pass.
At 8 concurrent requests the total is 622 tok/s; Q8_0 gives 143 tok/s single and 610 tok/s at 8. The user says llama.cpp's fastest two-GPU setup did about 100 tok/s on the same prompts, and that Geezuz averaged 164 tok/s and 214 on code at the same KL divergence. At 10K context it reports 159 tok/s single-request and 550 at 8 requests; vLLM still leads at 8 parallel requests by about 15%.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports Qwen3.8 Flash-Next at about 30 t/s average on an 8GB GPU, with peaks up to 48 t/s.
Setup is Strata on CachyOS with a 3-bit Unsloth quant, running at 42k context; 32k works fine and 42k leaves about 130MB of VRAM headroom.
The same model ran at 9 t/s before switching to Strata, and the user says 64k context does not fit in VRAM.