llamaperf

Qwen3.5 4B

on Unknown GPU

Tone: positive
Sep 26, 2026
Throughput
40-50 t/s gen

Use cases

multilingualtool-useagentic

Summary

User asks for recommendations on a small local model to replace Qwen3.5 4B as the reasoning layer of a local AI assistant. User reports getting around 40-50 tokens/sec with Qwen3.5 4B on unspecified limited hardware, and values that responsiveness. Priorities are general conversation, reasoning and instruction following, multilingual ability, context understanding, and reliable tool/function calling; the user is happy with the current model and wants a significant upgrade in the same memory and compute footprint.