| 🔢 all-MiniLM-L6-v2 | FP16 | 219 tok/s | 200 tok/s
+10%
| 0.1 GB | LocallyAI | editorial | 2026-03-07 |
Whisper Tiny | FP16 | 200 tok/s | 200 tok/s
+0%
| 0.1 GB | Whisper Transcription | editorial | 2026-01-18 |
| 🗣️ KittenTTS Nano | FP16 | 191 tok/s | 200 tok/s
-4%
| 0.1 GB | LocallyAI | community | 2026-04-06 |
| 🔢 BGE Small | FP16 | 189 tok/s | 200 tok/s
-5%
| 0.1 GB | LocallyAI | community | 2026-03-15 |
| 🗣️ KittenTTS Mini | FP16 | 179 tok/s | 184 tok/s
-3%
| 0.2 GB | LocallyAI | community | 2026-05-07 |
Wav2Vec2 Base | FP16 | 137 tok/s | 128 tok/s
+7%
| 0.3 GB | Whisper Transcription | community | 2026-01-20 |
SmolLM2 135M | FP16 | 109 tok/s | 109 tok/s
+0%
| 0.3 GB | LocallyAI | editorial | 2026-02-24 |
| 🔢 Nomic Embed Text | FP16 | 96 tok/s | 109 tok/s
-12%
| 0.3 GB | LocallyAI | editorial | 2026-01-04 |
| 🗣️ Kokoro 82M | FP16 | 75 tok/s | 82 tok/s
-9%
| 0.4 GB | LocallyAI | community | 2026-04-08 |
Ternary Bonsai 1.7B | Q2 | 72 tok/s | 77 tok/s
-6%
| 0.4 GB | LocallyAI | community | 2026-01-19 |
Whisper Small | FP16 | 53 tok/s | 60 tok/s
-12%
| 0.5 GB | Whisper Transcription | community | 2026-03-13 |
| 🔢 GTE Large | FP16 | 40 tok/s | 44 tok/s
-9%
| 0.7 GB | LocallyAI | community | 2026-02-09 |
Ternary Bonsai 4B | Q2 | 33 tok/s | 34 tok/s
-3%
| 0.9 GB | LocallyAI | community | 2026-02-08 |
SmolLM2 360M | FP16 | 32 tok/s | 32 tok/s
+0%
| 1.0 GB | LocallyAI | community | 2026-02-07 |
| 👁️ SmolVLM 500M | FP16 | 28 tok/s | 27 tok/s
+4%
| 1.2 GB | LocallyAI | community | 2026-02-11 |
| 🗣️ OuteTTS 0.3 500M | FP16 | 27 tok/s | 29 tok/s
-7%
| 1.1 GB | LocallyAI | community | 2026-01-01 |
Qwen 2.5 Coder 0.5B | FP16 | 25 tok/s | 25 tok/s
+0%
| 1.5 GB | LocallyAI | community | 2026-04-23 |
Bonsai 8B (1-bit) | Q1 | 24 tok/s | 25 tok/s
-4%
| 1.4 GB | LocallyAI | community | 2026-01-17 |
Qwen 2.5 0.5B | FP16 | 23 tok/s | 25 tok/s
-8%
| 1.4 GB | LocallyAI | community | 2026-03-17 |
Whisper Medium | FP16 | 22 tok/s | 20 tok/s
+10%
| 1.8 GB | Whisper Transcription | community | 2026-05-12 |
| 🗣️ NeuTTS Air | FP16 | 19 tok/s | 20 tok/s
-5%
| 1.8 GB | LocallyAI | community | 2026-03-17 |
| 🎙️ Distil-Whisper Large V3 | FP16 | 19 tok/s | 20 tok/s
-5%
| 1.8 GB | Whisper Transcription | community | 2026-04-21 |
Whisper Large V3 | Q8 | 17 tok/s | 18 tok/s
-6%
| 1.9 GB | Whisper Transcription | community | 2026-04-19 |
SmolLM2 1.7B | Q8 | 16 tok/s | 16 tok/s
+0%
| 2.0 GB | LocallyAI | community | 2026-05-28 |
Ternary Bonsai 8B | Q2 | 16 tok/s | 18 tok/s
-11%
| 1.9 GB | LocallyAI | community | 2026-04-09 |
Qwen 3.5 0.8B | FP16 | 16 tok/s | 16 tok/s
+0%
| 2.3 GB | LocallyAI | community | 2026-05-24 |
Qwen 2.5 Coder 1.5B | Q8 | 16 tok/s | 17 tok/s
-6%
| 2.0 GB | LocallyAI | community | 2026-01-20 |
Bonsai Image 4B | Q2 | 15 tok/s | 16 tok/s
-6%
| 2.3 GB | LocallyAI | editorial | 2026-01-28 |
Gemma 3 1B | FP16 | 14 tok/s | 13 tok/s
+8%
| 2.4 GB | LocallyAI | community | 2026-01-01 |
TinyLlama 1.1B | FP16 | 14 tok/s | 13 tok/s
+8%
| 2.5 GB | LocallyAI | editorial | 2026-02-10 |
StableLM 2 1.6B | Q8 | 14 tok/s | 16 tok/s
-12%
| 2.0 GB | LocallyAI | editorial | 2026-01-05 |
| 🧠 DeepSeek R1 Distill 1.5B | Q8 | 14 tok/s | 16 tok/s
-12%
| 2.2 GB | LocallyAI | community | 2026-04-22 |
| 👁️ Moondream 2B | Q8 | 14 tok/s | 15 tok/s
-7%
| 2.5 GB | LocallyAI | editorial | 2026-05-28 |
| 🎨 Stable Diffusion Turbo | FP16 | 14 tok/s | 14 tok/s
+0%
| 2.7 GB | LocallyAI | community | 2026-04-20 |
| 🎨 SDXL Turbo | Q4 | 14 tok/s | 13 tok/s
+8%
| 2.7 GB | LocallyAI | community | 2026-01-02 |
| 🗣️ Dia 1.6B | Q8 | 14 tok/s | 16 tok/s
-12%
| 2.0 GB | LocallyAI | editorial | 2026-01-05 |
Qwen 2.5 7B | Q2 | 13 tok/s | 12 tok/s
+8%
| 2.7 GB | LocallyAI | community | 2026-01-05 |
Qwen 3.5 2B | Q8 | 13 tok/s | 13 tok/s
+0%
| 2.8 GB | LocallyAI | community | 2026-04-21 |
Qwen 3.5 4B | Q4 | 13 tok/s | 12 tok/s
+8%
| 2.8 GB | LocallyAI | community | 2026-01-02 |
Qwen 2.5 Coder 3B | Q6 | 13 tok/s | 12 tok/s
+8%
| 2.8 GB | LocallyAI | community | 2026-02-09 |
| 👨💻 DeepSeek Coder 6.7B | Q2 | 13 tok/s | 13 tok/s
+0%
| 2.7 GB | LocallyAI | community | 2026-05-01 |
Qwen 2.5 Coder 7B | Q2 | 13 tok/s | 12 tok/s
+8%
| 3.2 GB | LocallyAI | community | 2026-01-17 |
LFM2.5 1.2B | FP16 | 12 tok/s | 13 tok/s
-8%
| 3.0 GB | LocallyAI | community | 2026-03-01 |
Gemma 3 4B | Q4 | 12 tok/s | 12 tok/s
+0%
| 2.8 GB | LocallyAI | community | 2026-01-03 |
Llama 3.1 8B | Q2 | 12 tok/s | 11 tok/s
+9%
| 3.0 GB | LocallyAI | community | 2026-01-05 |
Gemma 4 E2B | Q8 | 12 tok/s | 12 tok/s
+0%
| 2.8 GB | LocallyAI | community | 2026-03-14 |
Mistral 7B | Q2 | 12 tok/s | 13 tok/s
-8%
| 2.9 GB | LocallyAI | community | 2026-03-25 |
| 👨💻 StarCoder2 3B | Q6 | 12 tok/s | 12 tok/s
+0%
| 2.6 GB | LocallyAI | community | 2026-01-05 |
| 👁️ LLaVA 1.6 7B | Q2 | 12 tok/s | 13 tok/s
-8%
| 2.7 GB | LocallyAI | community | 2026-03-15 |
Llama 3.2 1B | FP16 | 11 tok/s | 11 tok/s
+0%
| 3.0 GB | LocallyAI | editorial | 2026-01-16 |
Qwen 2.5 3B | Q6 | 11 tok/s | 12 tok/s
-8%
| 3.1 GB | LocallyAI | community | 2026-03-24 |
Llama 3.2 3B | Q6 | 11 tok/s | 11 tok/s
+0%
| 3.3 GB | LocallyAI | community | 2026-05-24 |
| 🧠 DeepSeek R1 Distill 7B | Q2 | 11 tok/s | 12 tok/s
-8%
| 3.2 GB | LocallyAI | community | 2026-02-23 |
| 🧠 DeepSeek R1 Distill 8B | Q2 | 11 tok/s | 11 tok/s
+0%
| 2.9 GB | LocallyAI | community | 2026-01-05 |
Qwen 2.5 VL 7B | Q2 | 11 tok/s | 11 tok/s
+0%
| 2.8 GB | LocallyAI | community | 2026-01-06 |
Phi-4 Mini | Q5 | 10 tok/s | 11 tok/s
-9%
| 3.3 GB | LocallyAI | community | 2026-05-23 |
| 🧠 Qwen3 8B | Q2 | 10 tok/s | 11 tok/s
-9%
| 3.5 GB | LocallyAI | editorial | 2026-01-28 |
| 🎨 Stable Diffusion 3.5 Medium | Q8 | 10 tok/s | 11 tok/s
-9%
| 3.4 GB | LocallyAI | editorial | 2026-03-24 |