Ollama Setup Guide: Run Any LLM Locally in 10 Minutes

What is Ollama?

Ollama is the easiest way to run large language models locally. Think of it as Docker for AI models — you pull a model with one command, and it runs. No Python environment setup, no CUDA configuration, no cloud API keys.

Step 1: Install Ollama

Visit ollama.com and download the installer for your OS. Available for macOS, Windows, and Linux.

On Linux, you can install via the shell:

curl -fsSL https://ollama.com/install.sh | sh

Step 2: Pull a Model

Open a terminal and run:

ollama pull llama3.1:8b

This downloads the 8B parameter Llama 3.1 model (~5GB). Other good starting models:

• ollama pull mistral:7b — Fast and capable
• ollama pull phi3:mini — Tiny 3.8B, great on low RAM hardware
• ollama pull gemma2:9b — Google’s 9B model, excellent for tasks

Step 3: Start Chatting

ollama run llama3.1:8b

You will see a chat prompt. Type your message and press Enter. That is it — you are running AI locally.

Step 4: Connect a Chat UI (Optional)

Ollama runs a local API at http://localhost:11434. You can connect a chat interface like Open WebUI for a ChatGPT-style experience:

docker run -d -p 3000:8080 --add-host=host.docker.internal:host-gateway ghcr.io/open-webui/open-webui:main

Then open http://localhost:3000 in your browser.

FAQ

Does Ollama work on Windows? Yes, fully. Download the Windows installer from ollama.com.

Does Ollama use my GPU? Yes, automatically on NVIDIA and AMD GPUs. On Apple Silicon, it uses Metal for GPU acceleration.

How do I list installed models? Run ollama list in your terminal.