How to run an LLM locally in 5 minutes

Running an AI model on your own machine sounds intimidating. It isn’t. In 2026 you can have a private, offline chatbot running in about five minutes — no API key, no subscription, and nothing leaving your device. Here’s the fastest path.

Step 1: Check your hardware (30 seconds)

Almost any modern computer can run something. The question is how big. As a rule of thumb:

8 GB RAM → small chat and transcription models
16 GB RAM → most sub-4B models comfortably (see the list)
32 GB+ / a discrete GPU → 7B models and beyond

Not sure where you land? Run the in-browser device test — it estimates your capability without installing anything.

Step 2: Pick a tool

Two great starting points:

Ollama — a one-command tool. Best if you’re comfortable with a terminal.
LM Studio — a polished desktop app with a built-in model browser and chat UI. Best if you prefer clicking to typing.

Both are free. We’ll show Ollama because it’s the quickest.

Step 3: Install and run (the actual 5 minutes)

Download Ollama from its tool page, then open a terminal and run:

ollama run gemma4

That’s it. Ollama downloads the model the first time, then drops you into a chat prompt. Ask it anything — it’s running entirely on your machine.

Gemma 4 and Qwen 3.5 are the best small models in 2026. Want something smaller and faster? Try Qwen 3.5 2B. Got 16 GB+ and want stronger reasoning and long context? Try Qwen 3.5 9B:

ollama run qwen3.5

Step 4: Pick the right model for your machine

The model you run should match your hardware. Each model page shows which devices run it well, the best quantization, and estimated speed. If a model feels slow or won’t load, step down in size or quantization — a well-fitted q4 model beats a too-big one that swaps to disk.

What now?

Replace a paid tool: see Replace ChatGPT with local AI.
Go deeper on hardware: Best Mac for local AI.
Compare desktop apps: Ollama vs LM Studio vs Jan.

Welcome to local AI — private, free, and yours.