llamaperf

M2 Max 32GB vs NVIDIA Tesla P40 24GB for local LLMs

3 reports on the M2 Max 32GB and 12 on the NVIDIA Tesla P40 24GB

Which is faster for local LLMs?

On Qwen3.6 35B · 3B active at 4-bit, the M2 Max 32GB has 39.3 generation tokens/s (2 runs); prompt processing 671.4 tokens/s (one run); the NVIDIA Tesla P40 24GB has 9.7 generation tokens/s (one run). Those are community reports whose engines and settings differ, so the gap is a rough sign, not a controlled measure of the two cards. Memory bandwidth, which caps how fast a card can write, is 18% higher on the M2 Max 32GB (410 GB/s on the M2 Max 32GB, 346 on the NVIDIA Tesla P40 24GB). Across 2 shared models with generation measurements on both cards, the M2 Max 32GB's median is higher on 2 and the NVIDIA Tesla P40 24GB's on 0.

M2 Max 32GB
Memory
32GB unified
Memory bandwidth
410 GB/s
FP16 compute
13.6 TFLOPS
Reports
3
NVIDIA Tesla P40 24GB
Memory
24GB
Memory bandwidth
346 GB/s
FP16 compute
12 TFLOPS
Reports
12

Measured on both

Generation and prompt processing speeds from plain runs (one device, one request, no speculative decoding, the whole model in the device's memory) of the same model size at the same bit class. Each side shows the engines and reported context behind its median. Context can mean a configured window or prompt depth, so these are community comparisons with differing settings. How to read these

ModelM2 Max 32GBNVIDIA Tesla P40 24GB
Qwen3.6 35B · 3B active4-bit39.32 runsPP 671.4 (one run)llama.cpp, 1 not stated4K to 60K context9.7one runengine not statedcontext not stated
Ornith1.5 35B · 3B active4-bit108.0one runoMLXcontext not stated60.8one runPP 1104.0 (one run)llama.cppcontext not stated

Estimated by the calculator

The quant the calculator recommends for each card and its estimated speed at a 32,768-token context with 32 GB of system RAM, from memory bandwidth and the model's shape. Where a model was also measured above, the measurement wins.

ModelM2 Max 32GBNVIDIA Tesla P40 24GB
Qwen3.6 35B · 3B active76.1 t/sQ3_K_M44.4 t/sQ4_K_S
Ornith1.5 35B · 3B active76.1 t/sQ3_K_M44.4 t/sQ4_K_S
Qwen3.8 27B20.9 t/sQ4_K_S12.5 t/sQ4_K_M
Qwen3.8 125B · 6B activedoesn't fit at these settings16.9 t/sQ2_K, experts in RAM
DeepSeek V4 Flash 284B · 13B activedoesn't fit at these settingsdoesn't fit at these settings
DeepSeek V4.1 Flash 552B · 16B activedoesn't fit at these settingsdoesn't fit at these settings
Qwen3.6 27B20.9 t/sQ4_K_S12.5 t/sQ4_K_M
Gemma 4 26B · 4B active46.1 t/sQ4_K_M28.2 t/sQ5_K_S

Change the context, RAM or model in the calculator: M2 Max 32GB · NVIDIA Tesla P40 24GB

Every report on each card, with its source: M2 Max 32GB · NVIDIA Tesla P40 24GB

More comparisons