Qwen3.8 27B Bonsai-Llama-Jev
on NVIDIA RTX 3090 · llama.cpp
Sep 23, 2026
Use cases
tool-usevisionagentic
Summary
User reports Qwen3.8 27B in PrismML's Q2_64 quantization running at ~140 tok/s in chat on an RTX 3090, using under 10 GB VRAM including KV cache and vision projector.
Setup is a fork of PrismML's llama.cpp with a custom /v1/systemone endpoint for typed decisions, tested with 22,000 requests at massive parallelism; p50 latency ~171 ms including HTTP network latency.
User also cites ~143 tok/s for chat completion on an RTX 5090, and benchmark results of 76% Soft Accuracy vs ~88% for Jev-1.13 on a 22,000-request typed-decision benchmark.