llamaperf

Qwen3.6 35B (3B active)

on NVIDIA H100 80GB · Strata · 4,096 ctx

NVIDIA hardware
Nov 13, 2026
Throughput
290.0 t/s gen · 11486.0 tokens/s prompt processing (prefill)Prompt processing is how fast the input is read, not how long you wait for the first word. How to measure time to first token.
Quant
Q8_0 (GGUF)
VRAM reported
80 GB

Summary

User benchmarks Qwen3.6-35B-A3B at Q8_0 on an H100 80GB, reporting 290 t/s decode and 11,486 t/s prefill on a 4K prompt. Setup is the RustingStrata engine with Q8_0 weights (37.8 GB) fully in VRAM, CUDA kernels, no speculation for the reported figure; MTP with the model's own draft head reaches 508 t/s decode and costs 4-11% prefill. The post is a multi-GPU roundup: H100 NVL BF16 230 t/s decode, A100 80GB Q8_0 193 t/s, A100 80GB BF16 171 t/s, L40S 48GB Q8_0 165 t/s, all with MTP figures higher. Q8_0 on a 24 GB A30 with 63% of experts resident decodes at 130 t/s (200 with MTP), while BF16 at 31% resident reaches about 12.5 t/s. H100 prefill on Vulkan is 6,883 t/s against 11,486 on CUDA. No perplexity or KLD run exists on these Q8_0/BF16 files; the last check was UD-Q3_K_M on a 3060 at perplexity 9.0714 vs 9.0587 for llama.cpp, mean KLD 0.0055, same top token 96.3%.

Compare this run

Medians of plain runs: one device, one request, no speculative decoding, the whole model in the device's memory. The report above is one setup, so read it beside them and not as one of them.

Qwen3.6 35B · 3B active at 4-bit by hardware

Bars are medians on one scale, 0 to 215.1 tokens/s. Each dot is one run and opens its report. Press ▶ to watch text arrive at a row's speed.

Models most run on the NVIDIA H100 80GB

Bars are medians on one scale, 0 to 290.0 tokens/s. Press ▶ to watch text arrive at a row's speed.

Everything reported on the NVIDIA H100 80GB →Every Qwen3.6 report →Estimate Qwen3.6 on this card →