Qwen3.8 27B

on 2× AMD RX 7900 XTX · llama.cpp · 262,144 ctx

Sep 7, 2026

Throughput

27.0 t/s gen · 573.0 tokens/s prompt processing (prefill)Prompt processing measures input prefill speed; it does not tell you the time to first token.

Quant

Q6_K_XL (GGUF)

KV cache

VRAM reported

40 GB

Summary

User reports llama.cpp benchmarks on a dual-GPU setup of an RTX 4090 laptop and an RX 7900 XTX over TB4, reaching full 262k context with +70% generation and +40% prefill. Setup uses the CUDA backend for the NVIDIA card and Vulkan for the AMD card, with a q8_0 KV cache to enable 262k context at some cost in speed. MTP speculative decoding gave +50-70% generation but halved prefill, and the user filed a bug. A Q6_K_XL quant over Q8_0 gave +9% generation and more context on CUDA, and rebuilding to a newer llama.cpp gave +13% generation for free.

View source ↗