llamaperf
← All articles

Hardware planning

RTX Spark for local LLMs: what DGX Spark tells you

By llamaperf · · 7 min read

Quick answer

RTX Spark takes DGX Spark's design, a Grace CPU and a Blackwell GPU sharing up to 128 GB of memory, and puts it in Windows laptops and small desktops. Until owners measure one, DGX Spark's speeds are the best guide: around 59 tokens a second on a 120B mixture-of-experts model and under 30 on a dense 7B at 8 bits. How fast RTX Spark writes comes down to its memory bandwidth, which NVIDIA hasn't published.

What NVIDIA announced

A DGX Spark you can carry

RTX Spark is NVIDIA's first chip for Windows PCs. It pairs a 20-core Grace CPU with a Blackwell GPU that has 6,144 CUDA cores, and the two share up to 128 GB of memory. NVIDIA rates it at 1 petaflop of FP4 compute, the figure it also gives DGX Spark, and says laptops and compact desktops from ASUS, Dell, HP, Lenovo, Microsoft Surface and MSI arrive this fall (NVIDIA Newsroom). NVIDIA describes its local-model capacity as:

"run 120-billion-parameter large language models with 1 million tokens context" (NVIDIA Newsroom)

128 GB can hold a model that size. Whether it answers quickly is something the release doesn't cover.

The number NVIDIA left out

There's no memory bandwidth in the release, and for local models that's the figure you'd want first. Each token the model writes means reading its active weights out of memory again, so bandwidth caps writing speed. DGX Spark lists 273 GB/s on a 256-bit memory interface (NVIDIA DGX Spark). Laptop makers' spec sheets, as reported in a thread on NVIDIA's own forum, list faster LPDDR5X at 9,400 to 9,600 MT/s (NVIDIA developer forum). On a bus that stays 256 bits wide, that comes to roughly 300 GB/s. NVIDIA hasn't confirmed the bus width and that figure is our own arithmetic, so it stays a guess until someone measures a machine.

What DGX Spark already measures

Plain runs on the closest relative

The llama.cpp project keeps a set of llama-bench results for DGX Spark, all run without speculation (llama.cpp DGX Spark results). Until RTX Spark owners post their own, these are the closest guide you'll get.

ModelSize on diskReads a 2,048-token prompt (t/s)Writes (t/s)
gpt-oss 20B, MXFP411.27 GiB4,505.8283.43
gpt-oss 120B, MXFP459.02 GiB2,443.9158.72
Qwen3-Coder 30B-A3B, Q8_030.25 GiB2,986.9761.06
Qwen2.5-Coder 7B, Q8_07.54 GiB2,250.2829.43

The 30B model in the last two rows writes twice as fast as the 7B because it's a mixture of experts and reads only about 3B weights per token, while the dense 7B has to read all 7.54 GiB of itself every time.

Why dense models feel slow on it

Every machine in this class has the same problem with dense models. Take a 70B at about 4.5 bits a weight, which is roughly 40 GB. At 273 GB/s the GPU can read that about seven times a second at most, so in practice you'd expect fewer than seven tokens a second (that's an arithmetic example). A mixture-of-experts model with the same total size reads only a fraction of its weights per token and stays usable. Owners' DGX Spark reports, each with its model and quant, are on the leaderboard under the 128 GB unified memory tier, next to Strix Halo.

How it lines up against Strix Halo and the Mac

Same memory, different speeds

Compare RTX Spark with other machines offering around 128 GB of shared memory, using the bandwidth figures from our calculator.

MachineMemoryBandwidth
DGX Spark128 GB273 GB/s
AMD Strix Halo128 GB256 GB/s
Mac Studio, M5 Maxup to 128 GB614 GB/s
Mac Studio, M5 Ultra96 to 512 GB1,229 GB/s

On writing speed the Macs win easily, with two to four and a half times the bandwidth. Prompt reading depends on compute more than bandwidth, and DGX Spark already reads a 2,048-token prompt into a 120B model at over 2,400 tokens a second. It runs CUDA, which most inference software supports first, and it's the only one of these you can get as a Windows laptop. The CPU is Arm, though, so check that your tools have Windows on Arm builds before you buy.

Price is moving too

Memory costs have pushed all of these machines up. In February NVIDIA raised DGX Spark from $3,999 to $4,699, citing memory supply (NVIDIA developer forum). There's no RTX Spark price yet. When the laptops are listed, compare against what they sell for, and size your models in the calculator before you pay for any of them.

How to read the first RTX Spark numbers

Check what kind of run it was

Launch reviews and early owners will post the first speeds, and some will sit well above DGX Spark's. Before you believe the gap, check whether speculation was on. Multi-token prediction and draft models can double writing speed, and plenty of the Spark results posted so far had one of them switched on. Our speculative decoding guide explains when that speedup holds up for your own use.

Then look at how many requests ran at once, since a server total adds up every user. If a review quotes 200 tokens a second across eight streams, each person is getting about 25 (a made-up example). The guide to serving several users covers how to read those totals.

Laptops have one more variable

A laptop can throttle on battery or in a quiet power mode, and memory-bound work like writing tokens slows down with it. Give more weight to reviews that say whether the machine was plugged in and which power mode it ran in. Without those two details you can't tell whether a slow result came from the chip or from the power plan.

Frequently asked questions

Is RTX Spark the same chip as DGX Spark?

It uses the same idea in a PC: a Grace CPU and a Blackwell GPU share up to 128 GB of memory. NVIDIA rates it at 1 petaflop of FP4, just like DGX Spark. Whether it writes text as fast is still open, because NVIDIA hasn't published RTX Spark's memory bandwidth.

How fast will RTX Spark run a 70B model?

Slowly, if the model is dense. At about 4 bits a 70B is roughly 40 GB, and DGX Spark's 273 GB/s can read that about seven times a second at most, so expect single-digit tokens a second. A mixture-of-experts model of similar size runs several times faster.

Should I buy a Strix Halo laptop now or wait for RTX Spark?

Writing speed won't change much between them, since Strix Halo has 256 GB/s against DGX Spark's 273 GB/s. What RTX Spark adds is CUDA and NVIDIA's software. If your tools need CUDA and have Windows on Arm builds, wait for the first measurements. If they run on llama.cpp or Vulkan, Strix Halo does the same job today.

Is RTX Spark faster than a Mac Studio for local LLMs?

Not for writing text, which follows bandwidth. An M5 Max has more than twice the bandwidth and an M5 Ultra more than four times. RTX Spark holds its own on reading prompts, and it runs CUDA software the Mac can't.

What is the memory bandwidth of RTX Spark?

NVIDIA hasn't said. DGX Spark has 273 GB/s. Laptop spec sheets list faster LPDDR5X, which on the same 256-bit bus would come to roughly 300 GB/s, but that stays an estimate until someone measures a shipping machine.

Can RTX Spark run a 120B model?

The 128 GB versions can. DGX Spark runs gpt-oss 120B at about 59 tokens a second in llama.cpp, and it's that fast because gpt-oss is a mixture-of-experts model. The smaller RTX Spark laptops with 24 to 32 GB can't hold it.