DeepSeek R1 671B (37B active)
on Unknown GPU · llama.cpp
Summary
User reports DeepSeek-R1 671B at 4.2 t/s on a single AMD EPYC 7k62 (48 cores) with 8x64GB RAM, CPU only.
Setup is llama.cpp with Q5_K_S GGUF weights (461.81GB).
User also tested an AMD Threadripper Pro 3955WX (16 cores, 8x64GB RAM) at 2.8 t/s and dual AMD EPYC 7k62 (96 cores, 16x64GB RAM) at 2.9 t/s, and concludes llama.cpp is not optimized for dual-socket motherboards. User suggests a single-socket EPYC 9654 with 12 DDR5 channels and expects around 10 t/s.