By the end of this guide, Llama 3.1 8B will be answering questions from a terminal on your Windows machine. No cloud account, no API key, no WSL. Ollama does the heavy lifting: one installer, one pull command, and a local API on port 11434.

The numbers up front: the default llama3.1:8b model is a 4.9 GB Q4_K_M download that wants roughly 6.5 GB of VRAM at Ollama’s default 4,096-token context. That is the whole decision. The model fits any modern 8 GB card; the only real question is whether your hardware runs it on the GPU or falls back to the CPU.

Requirements

Windows 10 22H2 or later, 64-bit (Home or Pro).
NVIDIA: driver 551.61 or newer — that is Ollama’s Windows floor. The general NVIDIA floor is compute capability 5.0+ with a 550-series driver.
AMD Radeon: ROCm v7 / HIP 7, or a Vulkan-capable driver. On Windows, Vulkan is enabled by default — and it is the recommended path for RDNA2 cards (RX 6000 series), which may not expose ROCm v7 under current Windows drivers.
4 GB of disk for Ollama itself, plus room for models — downloads run from tens of GB to hundreds of GB depending on what you pull.
No administrator rights. Ollama installs into your user home directory by default.

1. Install Ollama on Windows

Two ways in. The PowerShell one-liner:

irm https://ollama.com/install.ps1 | iex

Or the GUI installer: download and run OllamaSetup.exe. A custom directory is a flag, not a dialog:

OllamaSetup.exe /DIR="d:\some\location"

After install, Ollama runs in the system tray in the background, and the ollama command works in cmd, PowerShell, or any terminal. As of October 2026 the latest release is v0.40.10; the Windows app auto-updates and shows “Restart to update” in the tray menu when a new version is ready.

Prefer the server without the tray app? The GitHub releases ship a standalone ollama-windows-amd64.zip; start it with ollama serve, and register it as a Windows service with NSSM if it must survive reboots.

2. Verify the server on port 11434

Open a terminal and run:

ollama ps

An empty table means the server is up and nothing is loaded. The API listens on http://localhost:11434 (bound to 127.0.0.1 by default). A one-line smoke test in PowerShell:

Invoke-WebRequest -Uri 'http://localhost:11434/api/generate' -Method POST -ContentType 'application/json' -Body '{"model": "llama3.1", "prompt": "Why is the sky blue?", "stream": false}'

A JSON answer means the stack works end to end.

3. Pull llama3.1:8b (4.9 GB, Q4_K_M)

ollama pull llama3.1:8b

That is a 4.9 GB Q4_K_M file — the same weights as llama3.1:latest. Models land in %HOMEPATH%\.ollama (i.e. C:\Users\<you>\.ollama). To move them, set the OLLAMA_MODELS environment variable, then restart Ollama from the Start menu.

Alternative quantizations from the model page:

Tag	Quant	Size	When it fits
`llama3.1:8b` (default)	Q4_K_M	4.9 GB	an 8 GB card; ~6.5 GB VRAM at 4k context
`llama3.1:8b-instruct-q3_k_m`	Q3_K_M	4.0 GB	a 6 GB card, or an 8 GB machine with no GPU
`llama3.1:8b-instruct-q8_0`	Q8_0	8.5 GB	a 12 GB card

The ~6.5 GB VRAM figure is one source’s estimate, not a measurement — treat it as a planning number, and check your own card in the VRAM calculator.

4. Start a chat and check ollama ps

ollama run llama3.1:8b

First run downloads the model, then drops you into a chat. Multi-line input: wrap the text in """. When you are in the chat, or in a second terminal:

ollama ps

The columns are NAME, ID, SIZE, PROCESSOR, UNTIL. In PROCESSOR you want 100% GPU; a mixed CPU/GPU box shows something like 48% CPU / 52% GPU. If the model is on the CPU, it will still answer — just about twenty times slower, as the next section shows.

The default context is 4,096 tokens. Change it in-session with /set parameter num_ctx 16384, or server-wide with the OLLAMA_CONTEXT_LENGTH environment variable. Models unload after 5 minutes idle (keep_alive); ollama stop <model> unloads immediately, and ollama run <model> "" preloads without sending a prompt.

5. Call the local API

Native API: POST /api/generate and POST /api/chat at http://localhost:11434.
OpenAI-compatible: point any OpenAI SDK at the /v1 endpoint with API key "ollama".
Other devices on your LAN: set OLLAMA_HOST=0.0.0.0 (or 0.0.0.0:11434) before starting the server.
Corporate proxy: set HTTPS_PROXY — the docs specifically say to avoid HTTP_PROXY.
Editor integration: the VS Code extension is Ollama.ollama.
Cloud features can be disabled entirely: OLLAMA_NO_CLOUD=1, or "disable_ollama_cloud": true in ~/.ollama/server.json — the log then shows Ollama cloud disabled: true.

What to expect

Speed depends on one thing: where the model runs. All figures below come from a single community benchmark — six DigitalOcean droplets, Ollama 0.32.9, llama3.1:8b Q4_K_M, Ollama’s own timing, minimum three runs per data point (August 2026):

Hardware	llama3.1:8b speed	Box cost
4 vCPU / 8 GB, no GPU	2.8 tok/s	$48/month
RTX 4000 Ada 20 GB	63 tok/s (1 user; 16 concurrent at reading speed)	$0.76/hour
H100	529 tok/s combined (32 concurrent)	—

Use the CPU row as your fallback expectation: 2.8 tok/s is readable for a single line, painful for a long answer. The GPU row is what the install is for.

Troubleshooting

could not connect to ollama service — the client cannot reach the server: it is not running, or you are hitting the wrong port. Start Ollama from the tray, confirm http://localhost:11434 responds, then retry.

No compatible GPUs were discovered — GPU discovery failed; the exact string was reported in GitHub issue #8674. Check the driver floor first (NVIDIA 551.61+, AMD ROCm v7/Vulkan), then the AMD GPU discovery section of the Ollama troubleshooting docs.

503 from the API — the request queue is full (default 512, OLLAMA_MAX_QUEUE). The server is overloaded, not broken; shed load or wait.

Progress bar renders as squares — old Windows 10 terminal fonts mangle the Unicode progress display. Cosmetic; switch fonts or ignore.

Slow networking with WSL2 on Windows 10 — disable Large Send Offload v2 on the “vEthernet (WSL)” adapter.

Debug mode — quit Ollama from the tray, then in PowerShell:

$env:OLLAMA_DEBUG="1"
& "ollama app.exe"

Logs live in %LOCALAPPDATA%\Ollama — server.log is the most recent, server-#.log the older ones. Look for the line Dynamic LLM libraries [rocm_v6 cpu cpu_avx cpu_avx2 cuda_v11 rocm_v5]: it lists the backends Ollama can actually use on your machine. To force a specific CPU library, OLLAMA_LLM_LIBRARY="cpu_avx2" ollama serve (cpu_avx2 is fastest, cpu the most compatible).

FAQ

Does Ollama on Windows need WSL? No. It is a native Windows app; NVIDIA and AMD Radeon GPUs work directly, and no administrator rights are required to install it.

Where does Ollama store models on Windows? %HOMEPATH%\.ollama by default. Set OLLAMA_MODELS and restart Ollama from the Start menu to move them. Note the uninstall path: Windows Settings → Apps → Add or remove programs — and if you changed OLLAMA_MODELS, the uninstall does not delete the models.

Why does my model disappear after a few minutes? Ollama unloads idle models after 5 minutes (keep_alive) to free VRAM/RAM. Extend it with OLLAMA_KEEP_ALIVE, or unload deliberately with ollama stop <model>.

Why only one model at a time on my AMD card? On Windows with AMD Radeon, Ollama limits you to one loaded model at a time — a ROCm VRAM-reporting limitation, not a choice you can tune away.

That covers the full Windows path. When your card is bigger than 8 GB, the Llama 3.1 8B on a 16 GB GPU guide picks up where this one stops: context limits, KV cache math, and the exact OOM errors. For the full model lineup, see the Local LLMs pillar.

Sources

Ollama docs — Windows — install without admin, driver floors, model and log paths, standalone CLI, uninstall behavior.
Ollama docs — FAQ — default context, OLLAMA_CONTEXT_LENGTH, keep_alive, ollama ps columns, OLLAMA_HOST, queue and concurrency limits, WSL2 networking fix.
Ollama docs — Troubleshooting — Windows log paths, OLLAMA_DEBUG, the Dynamic LLM libraries line, OLLAMA_LLM_LIBRARY override.
Ollama docs — CLI reference — exact command syntax: run, pull, ls, ps, stop, create, launch.
Ollama — llama3.1:8b model page — 4.9 GB Q4_K_M default tag.
ollama/ollama GitHub releases — v0.40.10 as of October 2026.
Ollama docs — install, GPU, OpenAI compatibility — the PowerShell install one-liner, NVIDIA compute-capability floor, /v1 endpoint with API key "ollama".
learnwithhasan — self-host Ollama benchmarks — the only measured numbers in this article: 2.8 tok/s CPU, 63 tok/s on RTX 4000 Ada, cost per million tokens (August 2026).
localaimaster — Llama 3.1 8B — the Q3_K_M/Q4_K_M/Q8_0 size table and the ~6.5 GB VRAM estimate.
ollama/ollama #8674 — “No compatible GPUs were discovered” failure report.

How to Run a Local LLM on Windows with Ollama