Shisa V2 Llama 3.3

generation:

4.7 tokens/s

prompt processing (prefill):

34.4 tokens/s

quant:

i1-Q4_K_M (GGUF)

flash attention:

on

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Shisa V2 Llama 3.3 at 4.7 t/s generation and 34.4 t/s prompt processing on a Ryzen AI Max+ 395 / Radeon 8060S under Linux. Setup is llama.cpp with the i1-Q4_K_M quant, HIP plus WMMA and FA, ngl 99, on one local ROCm device; the model is 39.59 GiB and 70.55B parameters. The 34.4 t/s is PP512 and the 4.7 t/s is a separate TG128 phase, not a same-request pair; context depth is unreported.

generation:

5.0 tokens/s

prompt processing (prefill):

77.3 tokens/s

quant:

i1-Q4_K_M (GGUF)

flash attention:

on

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Shisa V2 Llama 3.3 at 5 t/s generation and 77.3 t/s prompt processing on a Ryzen AI Max+ 395 / Radeon 8060S under Linux. Setup is llama.cpp Vulkan with the i1-Q4_K_M quant, full offload (ngl 99) and flash attention, on one local GPU. The figures are separate llama-bench phases: PP512 at 77.28 +/- 0.69 t/s and TG128 at 5.02 +/- 0.00 t/s, not a same-request pair. Context depth and batch size are unreported.