Skip to content

Open-Weight Models Keep Closing the Gap

industry By AI Hub Team · updated

For years the question “how far behind are open-weight models?” had an uncomfortable answer: nobody could measure it properly. Now there are numbers, and they are better than the industry narrative suggests.

Since January 2026, the most capable open-weight models have lagged frontier closed models by an average of four months — 8 points on the Epoch Capabilities Index, according to Epoch AI’s data insight. That is not a new gap; it is a stable one. In the previous analysis covering January 2023 through October 2025, the average lag was three months. Open weights stopped falling behind, and that changes the conversation.

Epoch adds a caveat worth repeating: the gap may be an underestimate. Open models are released alongside public benchmarks, so they tend to score well on public evals and worse on private ones. The real-world gap is probably a little larger than 4 months.

The NIST verdict: 8 months, and cheap

The most concrete measurement came from a government lab. In its evaluation published May 1, 2026, NIST’s Center for AI Security (CAISI) tested the open-weights DeepSeek V4 Pro and found it trailing the US frontier by about 8 months — the worst case of the cluster, but a useful ceiling.

The numbers behind that verdict:

BenchmarkDeepSeek V4 ProGPT-5.5
SWE-bench Verified74%—
GPQA-Diamond90%—
IRT Elo800 ± 281260 ± 28

Each ~200-point Elo step means the stronger model solves roughly three times as many tasks, so the 460-point gap is not a rounding error. What NIST also found: across seven benchmarks, V4 Pro ranged from 53% cheaper to 41% more expensive than GPT-5.4 mini. Price does not close a 460-point capability gap.

The three models that define the 2026 open frontier

DeepSeek V4 — the preview release (announced April 24, 2026) ships two open-weights models on HuggingFace under the MIT license:

  • V4 Pro: 1.6 trillion total parameters, 49 billion active (MoE), 1 million-token context — 8× the 128K of V3.2.
  • V4 Flash: 284 billion total, 13 billion active, same context.

Artificial Analysis put V4 Pro (Max) at 52 on its Intelligence Index, up from 42 for V3.2 — and ranks it the #2 open-weights reasoning model behind Kimi K2.6, which leads that list as of October 2026.

GLM-5.3 from Z.ai is the open-weights story in coding. Z.ai reports a 50% improvement over GLM-5.2 on its in-house Code Bench and open-source state of the art on Terminal Bench 3.0 and Agents’ Last Exam. The jump is real: GLM-5.2 scored 81.0 on Terminal-Bench, up from 63.5 for GLM-5.1. Notably, the weights were held back for two weeks after launch while safety evaluations ran — a governance step that used to be unique to closed labs. Epoch ranks GLM-5.3 as the top open-weight model on its cyber-capabilities index, trailing the closed frontier by 6–7 months in that domain.

Kimi K2.6 completes the picture: the leading open-weights reasoning model per Artificial Analysis, and the model that beats V4 Pro on the intelligence index. A 211-task coding study by Faros AI found the open GLM-5.2 + Kimi K2.6 pair statistically tied with — and cheaper and faster than — the frontier Claude Opus 4.8 and GPT-5.5 routes. For coding work, the open tier is the value leader.

What you can actually run locally

Here is where the story splits in two, and it matters for your wallet.

V4 Pro at 1.6T total parameters needs roughly a terabyte of memory at Q4. V4 Flash at 284B needs roughly 170 GB. Neither fits consumer hardware — the largest single card holds 32 GB. These models are an API story, not a home-lab story, and pretending otherwise is how people end up buying 512 GB of RAM for a “local AI dream.”

The models that do run locally — the 7B to 32B tier — are where the gap closing actually helps you. The VRAM math is stable: Q4 takes about 0.6 GB per billion parameters plus 20–30% for the KV cache.

ModelQ4_K_M sizeFits on
8B~4.9 GB8 GB cards
14B~9.3 GB16 GB cards, with context headroom
32B~20 GB24 GB cards
70B~43 GBdual 24 GB (e.g. 3090 + NVLink) or a Mac with 96 GB+

Every release cycle now makes this tier measurably smarter, so the “local model” you run in six months is not the same model as today’s. Epoch’s compute-trend analysis also suggests the first open model above 1e26 FLOPs of training compute lands by mid-2026 — the 7B–32B tier keeps getting distilled from bigger and bigger teachers.

And the frontier itself stays closed at the top: as of October 2026 the frontier leaderboard (cutoff 75.2) is entirely closed models — Claude Opus 5.5 (86.6), GPT-6.1 Sol (84.2), GPT-6 Astra (84), Claude Fable 5.1 (83.1), Gemini 4 Argon (82.8).

What we recommend

  • For coding via API: use the open tier — DeepSeek V4 Pro, GLM-5.3 or Kimi K2.6. The quality is frontier-adjacent and the price is a fraction of closed-flagship rates.
  • For local workloads: buy for the 14B–32B tier, not for models that do not fit. Check your card against real numbers with the VRAM Calculator, or read Estimate VRAM for an LLM if you want the method.
  • On an 8 GB card: start with Best 8B Models for 8 GB VRAM.
  • Hybrid is the default strategy: local models for daily volume, an open-weights API for the hard 10%. The 4-month gap makes the cloud leg cheap; the local leg keeps your data home.

Current open-weight reviews live in our Local LLMs pillar.

Sources