Phi-3 mini 4k instruct
NVIDIA RTX 3070 · LM Studio
- generation:
- 97.0 tokens/s
- quant:
- q4
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports 97 t/s generation. Source evidence (verbatim): Source: https://dev.to/fr4ncis/the-fastest-llama-uncovering-the-speed-of-llms-5ap8 # Original RTX3070 Phi3 mini4k instruct Q4 LMStudio measurement Francesco Mattia original experiments, September1,2024, edited September10. Selected single-prompt result: NVIDIA RTX3070 8GB, LMStudio, Phi3 mini4k instruct q4, approximately 97 tokens/s generation and 0.01s TTFT. Prompt: Who is the president of the US? Temperature0. Exact quantization subtype, tested repository/revision, runtime version, token counts, context depth, resident memory, speculation and concurrency unknown. Separate local PC CPU-only setup lists Ryzen5500 six cores and64GB RAM; whether those exact CPU/RAM specs apply to the selected GPU run is not established. The article also mixes Ollama benchmark-script and LMStudio tests; do not combine the earlier Phi3 input/output table with this specific q4 result. HF artifact link unavailable; do not guess. Original source: https://dev.to/fr4ncis/the-fastest-llama-uncovering-the-speed-of-llms-5ap8 . Selected structured generation measurement: approximately 97 tokens/s. Prompt processing speed unknown; do not derive it from TTFT. CPU/RAM belong to a separate CPU-only setup and must stay unknown for this selected GPU run.