Llama 3.1 8B Instruct
Llama 3.1 8B is the model we point beginners at first. It is the best balance of quality, speed and hardware requirements in its class, and every major runner (Ollama, LM Studio, llama.cpp, vLLM) supports it out of the box.
Who it is for: daily chat, summarization, light coding assistance, and the generation side of a local RAG pipeline. On a 16 GB GPU at Q8 it is a fast, capable assistant; at Q4 it squeezes onto 8 GB cards.
For bigger quality, step up to the 70B variant — but that needs a different tier of hardware (see Hardware).
Specifications
| Type | llm |
|---|---|
| Parameters | 8B |
| Context window | 128k |
| License | Meta Llama 3.1 Community License |
| VRAM recommended | 16 GB (Q8) GB |
| VRAM minimum | 8 GB (Q4) GB |
Pros
- Runs comfortably on 8 GB GPUs at Q4
- 128k context window
- Strong tool calling and function use
- Huge ecosystem of fine-tunes
Cons
- Weaker on long-horizon reasoning than larger models
- License requires renaming for commercial use above the MAU threshold