llamaperf

Ling-3.0 124B (5.1B active) Flash

on NVIDIA DGX Spark · llama.cpp · 262,144 ctx

NVIDIA hardware
Sep 11, 2026
Throughput
35.7 t/s gen
Quant
Q5_K_M (GGUF)
System RAM
128 GB

Use cases

long-context

Summary

User compares generation throughput on a 128 GB DGX Spark at three starting-context depths for two deployment paths. Official INT4 on a vLLM fork reaches 38.3 t/s short, 7.9 t/s at ~45K and 4.6 t/s at ~90K. Q5_K_M on llama.cpp reaches 35.7 t/s short, 33.6 t/s at ~45K and 33.2 t/s at ~90K. The config is 262,144 tokens of max context with ~103 GB total memory used, and streaming separates TTFT from decode. These are the creator's measurements, not an independent rerun, and two variables change at once (runtime and quantization), so no pure engine effect is isolated. The later 131,072-context serving default should not be attached to this table.