Strata and Qwen3.8-Flash-Next on one gaming GPU

One engine for one model

Qwen3.8-Flash-Next is Qwen's preview of its next architecture, and it's big. The model card lists 125B parameters with 6B active per token, a 51B n-gram embedding on top of that, and a 4B layer for multi-token prediction (MTP). Only a few billion parameters do any work on a given token. All of them still have to live somewhere, which is why Strata's standard download is about 70 GB and Unsloth's roughly 4-bit build is 94 GB.

Strata is an open-source engine (MIT licence) written for that one model. You run its installer on Windows or Linux, it picks a quant your RAM can hold, downloads it and serves an OpenAI-compatible API, plus an Anthropic-style one, from your own machine. It runs no other model, so every part of it can assume Flash-Next's layout. The README states the problem in two sentences:

"Models like this one usually run on servers with hundreds of gigabytes of graphics memory. Your graphics card has 12-24 GB." (Strata README)

How a 125B model fits on a 12 GB card

Flash-Next is a mixture-of-experts model with 512 experts per layer, and each token goes to 10 of them. Strata keeps the experts that get picked most on the GPU and a copy of every expert in system RAM. When a token needs one that isn't on the card, the CPU computes it on the spot while the GPU carries on with the rest. The n-gram table stays on the SSD and is read a few rows at a time. If that split between active and total parameters is new to you, our MoE guide explains why the first sets the speed and the second sets the memory.

Then there's speculation. Strata uses the model's own MTP layer to draft tokens ahead and checks them in one pass:

"a small helper guesses the next few words. The big model checks them all at once. You get the same answer, 1.6-1.8x sooner." (Strata README)

Setup switches that on for every launch. Most of the llama.cpp numbers you'll see posted for this model had nothing like it, so keep it in mind before you compare.

Strata's own numbers

The README measures two ordinary gaming PCs. One has an RTX 5070 with 12 GB, a six-core Ryzen 5 7600 and 64 GB of DDR5. The other pairs an RX 9070 XT (16 GB) with an older Ryzen 9 3900X and 47 GB of RAM, which by the README's own RAM table is too little for the IQ3 builds, so its column has gaps.

"Writes" is a short chat answer and "reads" is a 32K-token prompt. On the NVIDIA machine the Q2_0 row comes from engine 0.1.36 and the others from 0.1.26. Smaller quants write faster because each token pulls fewer bytes through the PC. Prompt reading runs into the thousands of tokens a second, and if you paste whole files into a coding agent you'll notice that long before you notice the writing speed. A card with more VRAM keeps more experts on the GPU, and the README expects a 24 GB RTX 3090 to write somewhere around 100 to 140 tokens a second.

A 16 GB card with 128 GB of RAM

One owner ran IQ3_S with 256K of context on Strata 0.1.39, an RTX 4070 Ti Super, a Ryzen 9 7950X3D and 128 GB of DDR5. Straight after install it wrote 8 tokens a second. After Strata's calibration step, which took ten minutes, it wrote 72 and read prompts at about 1,700 (r/LocalLLM). The same post ran Flash-Next through llama.cpp with offloading on that machine at 19 tokens a second, and the Strata build passed all 27 tasks of the owner's coding suite three runs out of three.

The jump from 8 to 72 is the part to remember. Strata's defaults were tuned on the six-core Ryzen, and a CPU with more cores, or with efficiency cores, can want different thread counts. So if your first run looks slow, calibrate before you decide anything.

llamaperf files Strata runs as running partly from system RAM, and as speculative unless the post says MTP was off (unsettled when it says nothing either way). Runs on two or more cards never count toward one card's figure. That keeps all of them out of the plain medians on GPU pages, so look for them in the report list, and add your own through submit.

Same card, same quant

The fairest comparison comes from the README's own RTX 5070 PC. A Reddit user ran the IQ3_XXS build on an RTX 5070 SFF with a Ryzen 5 7600 and 64 GB of DDR5, through llama.cpp and the FreeToken CLI. It wrote 13.8 tokens a second after a 1K prompt and read a 20K prompt at 104.7 (r/LocalLLaMA). That's the same card, CPU and RAM as Strata's test machine, and the poster later wrote Strata.

Strata's table shows IQ3_XXS on that PC writing at 62 tokens a second and reading a 32K prompt at 1,750. Writing comes out about 4.5 times as fast. Prompt reading comes out more than 16 times as fast, though the prompts differ in length (20K against 32K), so read that ratio loosely.

Where the gap comes from

Speculation explains some of it. The README credits MTP with 1.6 to 1.8 times on its own, and the llama.cpp run had none. Divide that back out of the 4.5 and Strata still writes about 2.5 to 2.8 times as fast on identical hardware, which is plain arithmetic on the published figures. The rest comes from where the experts live: the busiest ones stay on the GPU, and the CPU computes the ones the card doesn't hold. Prompts go through in chunks of up to 8,192 tokens at a time.

Generic partial offload in llama.cpp usually slows to a crawl once a model is bigger than your card, and our partial offload guide walks through why. Strata is what the same problem looks like when the engine is written around one model's layout. You pay for that in flexibility, because it runs Flash-Next and its builds and nothing else. Choosing an inference engine covers how to weigh that against a general engine.

Intel Arc and older cards

Strata started on NVIDIA and AMD, and the README now lists community builds, marked experimental, for older cards (Tesla P40 and V100, GTX 10 series, Radeon VII and MI50, RX 6700 XT and RX 5500 XT), for Intel Arc and for AMD's Ryzen AI Max, better known as Strix Halo. Arc has moved furthest. The 0.1.40.2 release notes call it officially supported on Linux, while the README still files it with the experimental builds, so expect rough edges either way. Those notes give a 32 GB Arc Pro B70 on IQ3_S 31 tokens a second on a story and 41 on code, with prompts read at 980 to 1,000. An 8 GB Arc A750 manages about 16 on IQ3_XXS. That's below the 12 GB the README asks for, and it still runs.

Two cards, or less RAM than the model

The README says two or three cards can share the model. The same release fixed how the automatic layer split chooses between placements that scored the same, and reports a 32K prompt reading at 2,797 tokens a second on four Radeon AI PRO R9700 cards (three of them went from 1,970 to 2,458). If you're on Linux with less RAM than the model needs, it made writing 20 to 42 percent faster by reading experts through the page cache. On a Tesla P100 box held to 32 GB, story writing went from 12.6 to 15.2 tokens a second.

RAM decides the quant

The README asks for an NVIDIA RTX 20 series card or newer, or one of a list of recent Radeons, with at least 12 GB of VRAM, plus 32 GB of RAM and about 80 GB of free disk. After that your RAM picks the quant. With 32 GB you get Coder, a build with half the experts removed that its authors measured at 91% of the full model's SWE-bench Verified score. It's weaker outside code, and that includes Chinese and other CJK text. 48 GB runs IQ2_XS or Q2_0. At 64 GB every standard size fits, and the installer recommends IQ2_XS there, with IQ3_S the best and slowest choice. 96 GB or more leaves room for Unsloth's roughly 4-bit UD-IQ4_XS. Unsloth's UD-Q4_K_XL comes closest to the full model, but on a 64 GB PC Strata reads most of it from the SSD and writes only 7 to 8.5 tokens a second.

The calculator treats Flash-Next as a mixture-of-experts model that streams its inactive experts from RAM, and it'll show what fits on your card at each quant. It assumes a general engine, so a Strata run should beat its estimate. The VRAM guide covers context, KV cache and system RAM in general.

Read any Strata number with its settings

Strata ships new builds every few days, and a number from last week can already be out of date. Write down the quant, the context length, whether calibration ran and the engine version beside every figure you keep. The 0.1.40.2 release alone moved the RTX 5070's IQ3_XXS story benchmark from 48.5 to 50.3 tokens a second. And since MTP is on, compare a Strata run with other speculative runs. Our speculative decoding guide explains how to read those.