Qwen3 8B

User reports Qwen3 8B at 89 t/s decode and 6.7k tok/s prefill on an RTX 5090 using PulsarLM, a new inference engine that evicts low-attention token pages to keep the context window bounded. Setup is PulsarLM with custom attention kernels that accumulate attention mass per token page and remove low-mass pages at intervals; RoPE embeddings are re-rotated to keep positions contiguous. Context snapshots avoid re-prefills at the cost of extra RAM. Batched decode reaches 1.2k tok/s at 16 concurrent requests. The engine is early-stage, tested only on Linux x86_64, and supports only smaller/older models; the author removed a BM25+attention recall mechanism and recommends pairing with external RAG.