Ling3-tiny
on NVIDIA RTX 5090 · 256,000 ctx
Use cases
long-context
Summary
User reports a KV cache blending technique that splits the prompt into chunks, caches each part in isolation, and concatenates the caches for decode.
The method maintains needle-in-haystack retrieval with overlap.
Prefill speed is boosted 3x to ~1.3k tps at 256k tokens, comparable to Qwen3.8-27b on an RTX 5090.