llamaperf

Qwen3.5 9B

on 2× SQRL FK33 · 256 ctx

Tone: positive
Oct 9, 2026
Throughput
3.2 t/s gen · 6.0 tokens/s prompt processing (prefill)Prompt processing measures input prefill speed; it does not tell you the time to first token.
Quant
INT4
VRAM reported
8 GB

Summary

User reports Qwen3.5-9B INT4 at about 3.2 t/s generation and 6 t/s prefill on 2x SQRL FK33 FPGAs at 75 MHz. Setup is a custom VHDL inference engine with a 256-token prompt, pipeline split across the two cards with the host carrying the residual between them; output was checked against llama.cpp layer by layer. Generation falls to about 2.4 t/s at 2-3k context and prefill to about 5.5 t/s on a 2.3k-token prompt. The user also gives modelled, not yet run, estimates for Qwen3.8-27B INT4 on Jungle Cat and 4x VU35P boards, and an ASIC feasibility estimate.