native PP512 TG128

Oct 13, 2026

Throughput

51.1 t/s gen · 173.5 tokens/s prompt processing (prefill)Prompt processing measures input prefill speed; it does not tell you the time to first token.

Quant

MXFP4 (GGUF)

KV cache

F16

Flash Attention

MTP (Multi-Token Prediction)

off

System RAM

128 GB

Summary

Historical native llama-bench measurement published by GitHub user apellegr, measured 2026-03-10 UTC and audited 2026-10-13. Exact source snapshot c97e6cfb6f8503823cda8a791c6f6751ac52daf1. Selected model: gpt-oss-120b, GGUF MXFP4. Native mean PP512=173.496458 tokens/s (sample SD 0.286975); native mean TG128=51.118165 tokens/s (sample SD 0.021166). Three samples per phase. These are separate synthetic PP512 and TG128 tests at n_depth=0, not generation after a 512-token prompt and not HTTP end-to-end throughput. No TTFT or request latency is measured. Hardware: raw cpu_info identifies AMD Ryzen AI Max+ 395 with Radeon 8060S; raw gpu_info is exactly Radeon 8060S Graphics. Source guide reports 16 physical Zen 5 CPU cores, 40 GPU compute units, 128GB shared LPDDR5X-8000. Chassis, power mode, clocks, thermal state, run isolation and background load are unspecified. Participating GPU count is exactly 1: the run's pinned 05728db llama-bench get_gpu_info() iterates all registered GPU/IGPU devices and joins each description with commas, without deduplicating; this output contains one device. The same revision's default auto selection selects available GPUs, or iGPUs if no other GPU exists. Raw devices=auto, backends=ROCm, ngl=999, n_cpu_moe=0 and no tensor overrides support one-device GPU-offloaded execution. The guide additionally states HIP_VISIBLE_DEVICES=0. main_gpu=0 or tensor_split=0.00 alone would NOT prove count: the latter formatter trims trailing zeros. This establishes one participating GPU, not a census of unexposed installed hardware. Runtime conflict: every selected raw PP/TG row reports build_commit=05728db and build_number=1. GitHub resolves that revision to 05728db18eea59de81ee3a7699739daaf015206b. Use this raw build identity. The guide instead claims Lychee build b8280, commit b2e1427 for all March 10 benchmarks; that inconsistent claim is not assigned to these rows. The referenced Lychee repository returned 404 during audit. The actual fork/binary hash and any uncommitted changes remain unknown. Guide-reported surrounding stack is Ubuntu25.10, kernel6.17.0-14-generic, ROCm7.2.0, HIP7.2.26015.70200-43, rocBLAS5.2.0.70200-43, Clang22.0.0/GCC15.2.0 and gfx1151 target; these ancillary versions are not independently in raw rows. Raw settings: n_batch=512, n_ubatch=512, n_threads=16, K/V=f16, flash_attn=true, use_mmap=false, use_direct_io=false, no_kv_offload=false, split_mode=layer, n_gpu_layers=999, n_cpu_moe=0, embeddings=false, no_op_offload=0, no_host=false. PP n_prompt=512/n_gen=0; TG n_prompt=0/n_gen=128; both n_depth=0. Benchmark context request in pinned code is n_prompt+n_gen+n_depth, separately512/128 before runtime padding, not the model's supported context capacity. No long-context rate is implied. The guide's command specifies -r3; raw includes all3 samples. Pinned code performs default warmup before timed repetitions, clears KV state each repetition, uses synthetic/random tokens (BOS when appropriate), and synchronizes GPU execution. Exact full argv/no-warmup override is not logged. The elapsed timer is std::chrono::high_resolution_clock, with timestamps represented in nanoseconds and elapsed=end-start. Throughput is the arithmetic mean of per-sample token_count*1e9/elapsed_ns, verified from raw nanoseconds; it is not token_count divided by mean latency. This is a microbenchmark, not a language-quality or real-conversation test. PP timestamp 2026-03-10T20:31:39Z; elapsed ns samples [2956521834, 2949616216, 2947084768]. TG timestamp 2026-03-10T20:31:51Z; elapsed ns samples [2505184557, 2503577529, 2503245480]. Preserve raw rates rather than the guide's rounded values. The wrapper's quant=f16 conflicts with raw model_type=gpt-oss 120B MXFP4 MoE; f16 matches K/V dtype and must not be treated as weight quantization. Raw model_size=63374323968 is the runtime model/tensor payload byte count; wrapper model_size_gb=59.0 is a size label, not measured peak RAM. Full HF GGUF file bytes also include metadata and can differ from payload bytes. No artifact hash match is inferred from these sizes. Do not enter them as memory usage. System128GB is one shared pool, not128GB RAM plus128GB discrete VRAM. Guide GTT limit and approximate memory estimates are not per-run measured allocation; GPU reservation and peak memory remain unknown. Artifact provenance: raw basename gpt-oss-120b-mxfp4-00001-of-00003.gguf matches the pinned source downloader's ggml-org/gpt-oss-120b-GGUF destination and quant. The exact tested filename has three lowercase mxfp4 shards. The verified historical HF reference revision d932fcea62f83e088d8f076a2cd2d7eb02dfa682 predates the run. The repository history records deleting these shards on 2026-07-16; current main contains a different, single uppercase MXFP4 file. Use the historical three-shard link, never silently substitute current main. HF metadata verified exact file(s) at reference revision d932fcea62f83e088d8f076a2cd2d7eb02dfa682; this is an independently verified artifact reference, NOT a proven tested-weight revision/hash. The run supplies no HF revision, weight hash or download log. The exact measured publisher is attributed through the matching source downloader, not established from raw filename alone. Base/reference model is openai/gpt-oss-120b; it must not replace the GGUF artifact link. Only public text and metadata were read, with no model weights downloaded or source programs run. Sources: Raw: https://github.com/apellegr/Strix-Halo-Models/blob/c97e6cfb6f8503823cda8a791c6f6751ac52daf1/benchmarks/full_benchmark_20260310_183313.jsonl#L2993-L3080 Guide: https://github.com/apellegr/Strix-Halo-Models/blob/c97e6cfb6f8503823cda8a791c6f6751ac52daf1/docs/MODELS.md Downloader: https://github.com/apellegr/Strix-Halo-Models/blob/c97e6cfb6f8503823cda8a791c6f6751ac52daf1/scripts/setup/download_strix_halo_models.sh#L296-L302 Pinned GPU enumeration: https://github.com/ggml-org/llama.cpp/blob/05728db18eea59de81ee3a7699739daaf015206b/tools/llama-bench/llama-bench.cpp#L127-L136 Pinned device selection: https://github.com/ggml-org/llama.cpp/blob/05728db18eea59de81ee3a7699739daaf015206b/src/llama.cpp#L912-L979 Pinned timer: https://github.com/ggml-org/llama.cpp/blob/05728db18eea59de81ee3a7699739daaf015206b/tools/llama-bench/llama-bench.cpp#L35-L38 Pinned timing/metric code: https://github.com/ggml-org/llama.cpp/blob/05728db18eea59de81ee3a7699739daaf015206b/tools/llama-bench/llama-bench.cpp#L1378-L1392 and #L1962-L2012 and #L2159-L2258 Exact GGUF reference: https://huggingface.co/ggml-org/gpt-oss-120b-GGUF/blob/d932fcea62f83e088d8f076a2cd2d7eb02dfa682/gpt-oss-120b-mxfp4-00001-of-00003.gguf Full artifact tree: https://huggingface.co/ggml-org/gpt-oss-120b-GGUF/tree/d932fcea62f83e088d8f076a2cd2d7eb02dfa682 Base/reference: https://huggingface.co/openai/gpt-oss-120b Tensor payload size: https://github.com/ggml-org/llama.cpp/blob/05728db18eea59de81ee3a7699739daaf015206b/src/llama-model-loader.cpp#L555-L566 Structured PP/TG are rounded to the form’s 0.1 tokens/s step; exact source rates remain above.