Local model performance on your hardware

generation:

14.0 tokens/s

quant:

Q4_K_S (GGUF)

User compares DanilAI and LM Studio running Qwen3-30B-A3B-Instruct-2507 on an RTX 4050 Laptop (6 GB VRAM) with a Ryzen 7 7435HS and 24 GB RAM. LM Studio with Q4_K_S quantization reached 14.03 tokens/sec; DanilAI with hybrid FP16/8-bit/4-bit precision plus DanilDrive compression reached 25.0 tokens/sec. A separate test ran Qwen3-0.6B in FP16 with DanilDrive, reporting approximately 45% lossless compression. OBS Studio was recording throughout and may have affected performance.

generation:

76.2 tokens/s

User reports Qwen3-0.6B at 76.2 t/s on an RTX 4050 Laptop 6GB, a 4.19x speedup over uncompressed bf16 (18.2 t/s). Setup is a custom engine (DanilDrive-W) with lossless dictionary-compressed bf16 weights, decompressed inline in the CUDA kernel; file size drops from 1433.7 MB to 765.5 MB (-46.6%). User also tested Qwen3-Coder-30B-A3B against Ollama and LM Studio on the same Q4_K_M GGUF, getting 16-18 t/s versus their 24-26 t/s, and notes greedy decoding can diverge after ~200 characters due to float summation order.

generation:

129.0 tokens/s

prompt processing (prefill):

8500.0 tokens/s

User benchmarks 20 small models on an RTX 4050 6GB. Models tested include Granite 4.1 3B, Gemma-4-agentic-e2b, Nemotron-3-Nano-4B and LFM2.5-8B-A1B. The user selects LFM2.5-1.2B-Instruct as a cheap always-on model.