llamaperf

Qwen3.8 27B Bonsai-Llama-Jev

on NVIDIA RTX 3090 · llama.cpp

Tone: positiveNVIDIA hardware
Sep 23, 2026
Throughput
140.0 t/s gen
Quant
Q2_64 (GGUF)
VRAM reported
10 GB

Use cases

tool-usevisionagentic

Summary

User reports Qwen3.8 27B in PrismML's Q2_64 quantization running at ~140 tok/s in chat on an RTX 3090, using under 10 GB VRAM including KV cache and vision projector. Setup is a fork of PrismML's llama.cpp with a custom /v1/systemone endpoint for typed decisions, tested with 22,000 requests at massive parallelism; p50 latency ~171 ms including HTTP network latency. User also cites ~143 tok/s for chat completion on an RTX 5090, and benchmark results of 76% Soft Accuracy vs ~88% for Jev-1.13 on a 22,000-request typed-decision benchmark.