Qwen3.8 27B

User benchmarks Qwen3.8-27B NVFP4 at 29.3 tok/s single-stream decode on one Tesla V100-PCIE-32GB, comparing 1Cat-vLLM v1.5.1 against llama.cpp. Setup is 1Cat-vLLM with NVFP4 weights and fp8_e4m3 KV cache at 32K context, tensor parallel 1, 29.9 GB VRAM; llama.cpp ran the same model as UD-Q4_K_M GGUF with q8_0 KV cache at 131072 context, 20.6 GB VRAM. 1Cat is about 10% behind llama.cpp on single-stream decode (32.7 tok/s) but 45% ahead with 4 concurrent requests (110 tok/s aggregate vs 75.7) and reads long prompts 25-42% faster (1055 tok/s at 2.9K down to 847 at 29.4K). A separate Qwen3-8B FP16 run on the same card gave 40.7 tok/s single-stream and 141 tok/s at 4 concurrent for 1Cat, against 49.4 and 155 for llama.cpp.