Qwen3.8 27B Bonsai 2
on NVIDIA RTX 3090 · llama-cpp-prism · 10,000 ctx
Summary
User reports Qwen3.8 27B Bonsai 2 at PQ2_0 (2.13 bpw, 7.21 GB) on one RTX 3090 with the llama-cpp-prism ternary fork, prefilling at 834.8 t/s at 10K.
Setup holds the full 262144 context on the single card at 17.1 GB with q8_0 KV; the quant drops the MTP head, so the run is un-drafted. Decode is given per workload only, 42.76 t/s narrative and 41.01 code, with 141 ms TTFT.
The same harness PR adds an exl3/TabbyAPI engine on the 2x 3090 reference rig, whose CPU-computed MoE slugs decode at 50.5 and 39.3 T/s at 204800 context with Q4 KV.