Qwen3.6 27B
on NVIDIA RTX 3090 · Luce DFlash (custom C++/CUDA stack on ggml) · 256,000 ctx
Use cases
codingmath
Summary
User reports speculative decoding with DFlash on a single RTX 3090, targeting Qwen3.6-27B Q4_K_M GGUF at ~16 GB with a z-lab Qwen3.6-27B-DFlash bf16 draft at ~3.46 GB.
Setup is DDTree tree-verify with block size 16, budget 22, and greedy verify. The KV cache is compressed to TQ3_0 at 3.5 bpv, roughly 9.7x versus F16, with a 4096-slot ring buffer enabling 256K context in 24 GB. Decode uses sliding-window flash attention with a 2048-token window, and prefill ubatch auto-bumps from 16 to 192 for prompts over 2048 tokens. The endpoint is OpenAI-compatible and CUDA only, with no Metal, ROCm, or multi-GPU support.
Output is bit-identical to autoregressive in AR mode, and the draft matches the z-lab PyTorch reference at cos sim 0.999812.