NVIDIA RTX 4090

The RTX 4090 is the fastest 24 GB card on the market: 1,008 GB/s of GDDR6X bandwidth, and bandwidth is what drives tokens per second. It is also the most expensive 24 GB you can buy — used. New stock is scarce and used cards go for ~$2,000+ as of October 2026.

What it runs well:

• 7–8B models at Q4 — 55–95 tok/s in one hardware guide (June 2026)
• 32B models at Q4 — 12–20 tok/s; the 20 GB Q4_K_M file fits with 4–6 GB left for context
• SDXL-class image models at full resolution

The claim you will hear about it: “a 4090 runs 70B.” It does not. Ollama’s llama3.3:70b downloads as 43 GB at Q4_K_M, and 24 GB cannot hold it — offloading that many layers to CPU RAM collapses the speed. For 70B you need two 24 GB cards pooled with NVLink (48 GB, the dual-3090 route) or a Mac with at least 96 GB of unified memory.

Trade-offs: 450 W under load (850 W PSU minimum, 1000 W is safer), three fans, and a price the 2026 GPU market has not helped. If 70B is the goal, skip the 4090 — the ranked GPU list covers the dual-3090 and Mac Studio routes.

Verdict: the pick when 32B-class models must run fast and the RTX 5090 (32 GB, street $3,000–5,000 as of October 2026) is off the table. See the VRAM calculator to confirm what fits, and the hardware pillar for the rest of the lineup.