RTX 3080 20GB

reported speed:

25.0 tokens/s generation · 870.0 tokens/s prompt processing

quant:

4.05 EXL3

Reported by the source; GPU count, offloading and concurrent requests can change this figure. Check the full setup before comparing.

CPU-offloaded inference with 128GB system RAM. Compared to llama.cpp: 3.2x faster prefill, 2x faster decode. Also tested GLM 5.3 Flash with 3.05 EXL3, which ran 2x slower in decode than llama.cpp. Decode speeds warm up over time.

reported speed:

17.5 tokens/s generation · 273.2 tokens/s prompt processing

quant:

IQ2XXS-w2Q2K-AProjQ8-SExpQ8-OutQ8 (GGUF)

kv:

q8_0

Reported by the source; GPU count, offloading and concurrent requests can change this figure. Check the full setup before comparing.

Running on 2x RTX 3080 20GB with 64GB DDR5. Uses llama.cpp fork by fairydreaming. Achieves 17.53 tk/s generation and 273.21 tk/s prefill at 128k context. Model is 86.7 GB. Uses --n-cpu-moe 28 and split-mode layer.

reported speed:

57.5 tokens/s generation

quant:

Q6_K_L (gguf)

Reported by the source; GPU count, offloading and concurrent requests can change this figure. Check the full setup before comparing.

Model is actually vectionlabs/Salience-27B-R5-GGUF, not Qwen 3.8. The post title mentions Qwen 3.8 27B but the setup says bartowski/vectionlabs_Salience-27B-R5-GGUF. The model was used to build a Tibia clone in HTML/CSS/JS. Speed reported as 55~60 tk/s, averaged to 57.5. Uses MTP and vision.

reported speed:

20.0 tokens/s generation

quant:

Q4_K_M (gguf)

Reported by the source; GPU count, offloading and concurrent requests can change this figure. Check the full setup before comparing.

User is planning to buy a GPU to run Qwen 3.8 27B Q4_K_M at +20 T/s. Mentions RTX 3080 20GB, AMD Mi50 32GB, and modified 2080 Ti 22GB as options. No benchmark results yet.

reported speed:

11.5 tokens/s generation · 300.0 tokens/s prompt processing

quant:

IQ2_M (gguf)

Reported by the source; GPU count, offloading and concurrent requests can change this figure. Check the full setup before comparing.

First local model to achieve 100% on SQL benchmark. Custom IQ2_M GGUF with grafted tensors from antirez GGUF, running on modified ds4 engine. Mainline llama.cpp gives 100pp and 8tg. Dual RTX 3080 20GB with 96GB RAM and 9800X3D.