Ling-3.0

generation:

15.0 tokens/s

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.

User reports Ling 3.0 Tiny at 15 t/s on an AMD 680M iGPU with llama.cpp, in power saver mode. Gemma 4 E4B ran at 9 t/s on the same iGPU, also in power saver mode; the laptop stayed cold and silent. User calls Ling usable and asks whether anyone has run Gemma 4 26B-A4B on a 780M laptop, guessing around 20 t/s by extrapolation.

generation:

73.1 tokens/s

quant:

oQ5 affine mixed precision, group 64 (MLX)

mtp (multi-token prediction):

off

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.

User reports Ling-3.0 Flash at 73.1 t/s on an M3 Ultra 256GB. Setup is oMLX with oQ5 affine mixed precision, group 64, on a 124B total / 5.1B active model. The figure is the median of three greedy 256-token runs (73.23, 73.10, 73.09) after a 256-token warm-up, with a 39-token prompt; peak memory is reported at 87.44 GB.

generation:

10.0 tokens/s

quant:

Q6 (GGUF)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.

User reports Ling 3.0 Tiny at about 10 tokens per second on a 2017 laptop with a 7th gen i5 and 8GB of RAM. Setup is llama.cpp with the first Q6 quant found, CPU-only with no GPU or VRAM. The model ran an agentic loop writing, running and iterating on a network-scanning script, finishing in about 20 minutes. User notes one billion active parameters is small enough for an old CPU to run at this speed with no optimization effort.

generation:

54.0 tokens/s

prompt processing (prefill):

245.0 tokens/s

quant:

Q4_K_M (GGUF)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User benchmarks Ling 3.0 tiny against Gemma 4 26B-A4B for batched JSON speaker attribution on an NVIDIA T1000 8GB. Ling 3.0 tiny runs at 245 t/s prefill and 54 t/s decode, 1.8x faster end to end than Gemma. Setup is llama.cpp with the official Q4_K_M GGUF at 16384 context, all 25 layers on the GPU, no speculative decoding. Gemma uses UD-Q4_K_XL with experts in system RAM and an MTP draft head. Ling scores 56.9% exact speaker match versus Gemma's 94.6% across 485 lines, with narrator swaps and refusals. User is not adopting it.

generation:

35.7 tokens/s

quant:

Q5_K_M (GGUF)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.

User compares generation throughput on a 128 GB DGX Spark at three starting-context depths for two deployment paths. Official INT4 on a vLLM fork reaches 38.3 t/s short, 7.9 t/s at ~45K and 4.6 t/s at ~90K. Q5_K_M on llama.cpp reaches 35.7 t/s short, 33.6 t/s at ~45K and 33.2 t/s at ~90K. The config is 262,144 tokens of max context with ~103 GB total memory used, and streaming separates TTFT from decode. These are the creator's measurements, not an independent rerun, and two variables change at once (runtime and quantization), so no pure engine effect is isolated. The later 131,072-context serving default should not be attached to this table.

generation:

40.9 tokens/s

quant:

INT4

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.

User benchmarks Ling-3.0-flash on a DGX Spark at 40.9 t/s on a short coding task. Setup is CUDA graphs with MTP n=1. Prose throughput is 38.7 t/s at 512-token output and 37.3 t/s at 2048-token output. MTP n=2 and n=3 are slower for prose. Acceptance lengths are 1.87 at n=1, 2.39 at n=2 and 2.77 at n=3. Without MTP the baseline is 22.9 t/s with CUDA graphs and 20.8 t/s eager.

generation:

40.2 tokens/s

quant:

Q5_K_M (GGUF)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.

User reports Ling-3.0-flash at 40.2 tok/s single stream decode with the Q5_K_M quant on one DGX Spark, from benchmarks posted by sudoingX on X. Setup is the community GGUF quant ladder, with 5.1B active out of 124B parameters. Q4_K_M reaches 38.2 tok/s and Q6_K 32.0 tok/s. DeepSeek V4 Flash measured 16.5 tok/s on the same box.

generation:

38.7 tokens/s

quant:

INT4

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.

User reports Ling-3.0-flash (124B) at 38.7 tok/s on the official INT4 quant on one DGX Spark, from benchmarks posted by sudoingX on X. The community GGUF reaches 35.2 tok/s, and the result is 2.4x what DeepSeek V4 Flash does on the same machine. The benchmarker first posted that the official quants do not run on a single Spark, then corrected that two days later.