NVIDIA T1000 8GB reports

generation:

54.0 tokens/s

prompt processing (prefill):

245.0 tokens/s

quant:

Q4_K_M (GGUF)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User benchmarks Ling 3.0 tiny against Gemma 4 26B-A4B for batched JSON speaker attribution on an NVIDIA T1000 8GB. Ling 3.0 tiny runs at 245 t/s prefill and 54 t/s decode, 1.8x faster end to end than Gemma. Setup is llama.cpp with the official Q4_K_M GGUF at 16384 context, all 25 layers on the GPU, no speculative decoding. Gemma uses UD-Q4_K_XL with experts in system RAM and an MTP draft head. Ling scores 56.9% exact speaker match versus Gemma's 94.6% across 485 lines, with narrator swaps and refusals. User is not adopting it.

generation:

6.0 tokens/s

prompt processing (prefill):

18.0 tokens/s

quant:

PTQ1_0 (GGUF)

kv:

q8_0

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User benchmarks Ternary Bonsai 2 27B at ~6 t/s generation and ~18 t/s prefill on an NVIDIA T1000 8GB, comparing it against Gemma 4 26B-A4B for audiobook speaker attribution. Setup is a PrismML llama.cpp fork with PTQ1_0 ternary weights (5.95 GB), q8_0 KV cache, 8192 context, and all layers on GPU; the model scores 448/480 (93.3%) exact speaker matches on four chapters. The user does not adopt it, citing narrator identity drift, adjacent-turn swaps, dropped lines, and an intonation regression at 5-10x the wall-clock of the incumbent Gemma 4 26B-A4B MoE.