Local model performance on your hardware

generation:

25-35 tokens/s

quant:

Q5_K_M (GGUF)

User reports Qwen3.6-35B-A3B at 25-35 tok/s on an RTX 4050 Laptop 6GB with 64GB DDR5 RAM and an i5-13420H. Setup is LM Studio with Q5_K_M and RAM offload, running 34-36k context windows. User notes a 2-minute time to first token and says the speed is outstanding for the hardware. They ask whether the model and quant are a good fit for a 3D model parameter data structuring workflow, and mention they do not need uncensored or abliterated variants.

generation:

14.0 tokens/s

quant:

Q4_K_S (GGUF)

User compares DanilAI and LM Studio running Qwen3-30B-A3B-Instruct-2507 on an RTX 4050 Laptop (6 GB VRAM) with a Ryzen 7 7435HS and 24 GB RAM. LM Studio with Q4_K_S quantization reached 14.03 tokens/sec; DanilAI with hybrid FP16/8-bit/4-bit precision plus DanilDrive compression reached 25.0 tokens/sec. A separate test ran Qwen3-0.6B in FP16 with DanilDrive, reporting approximately 45% lossless compression. OBS Studio was recording throughout and may have affected performance.

generation:

129.0 tokens/s

prompt processing (prefill):

8500.0 tokens/s

User benchmarks 20 small models on an RTX 4050 6GB. Models tested include Granite 4.1 3B, Gemma-4-agentic-e2b, Nemotron-3-Nano-4B and LFM2.5-8B-A1B. The user selects LFM2.5-1.2B-Instruct as a cheap always-on model.