DeepSeek · 6 reports
Unknown GPU
The post describes an experiment comparing a baseline (Claude Code + DeepSeek V4) with an Attemory retrieval-hint setup. It reports a 43.8% token drop and a judge score tie (83.39 vs 83.17) on SWE-QA. No hardware or inference engine details are mentioned.
M1 Ultra 128GB · LM Studio
Reported by the source; GPU count, offloading and concurrent requests can change this figure. Check the full setup before comparing.
Patched engine improved speed from 5-6 to 15-16 tok/s. Wired limit at 120GB. Unsloth UD-IQ3_XXS quant.
3× RTX 3090 · llama.cpp · 1,000,000 ctx
Reported by the source; GPU count, offloading and concurrent requests can change this figure. Check the full setup before comparing.
3x3090 with DDR5, 1MM lossless context. 4th 3090 runs gemma 12b and flux2klein. Cold start 15tg, steady 10 TG, 300PP degrades to 100PP after 64000 tokens. llama.cpp fork with deepseek4 arch support.
Unknown GPU
Reported by the source; GPU count, offloading and concurrent requests can change this figure. Check the full setup before comparing.
User left DeepSeek V4 running at 9 t/s on Windows; system went to sleep despite settings, causing hardware failure. No hardware details provided.
Reported by the source; GPU count, offloading and concurrent requests can change this figure. Check the full setup before comparing.
Custom Turing CUDA kernels for W8A8 INT8 matmul. Heterogeneous inference with 4x 11/22GB VRAM and 1TB system RAM. Computation-communication overlap for MoE routing. Open-sourced on GitHub.
2× Radeon AI PRO R9700 32GB · 16,384 ctx
Reported by the source; GPU count, offloading and concurrent requests can change this figure. Check the full setup before comparing.
Lucebox custom setup with AMD Radeon AI PRO R9700 and Strix Halo 128GB. Asymmetric parallelism: R9700 handles dense path, hot experts, cache, draft model; Strix Halo holds other experts. 3.63x faster than single DGX Spark. Experimental, q2 quant, 16k context. Working on KVFlash for 64k-128k.