VRAM

VRAM (video RAM) is the dedicated high-bandwidth memory on a graphics card. When running local LLMs, the model’s weights, the KV cache (context) and intermediate activations all live in VRAM.

Why it matters more than compute: a model that does not fit in VRAM cannot run, full stop — no amount of FLOPS fixes that. A card that is 30% slower but has double the VRAM is usually the better local-AI purchase.

Rule of thumb: weights at Q4 need ~0.6 GB per billion parameters, plus ~1 GB overhead, plus more for long context. See the VRAM Calculator.

Related: Quantization, GPU.