Skip to content
← All news

News1 min read

#1 on Lean-Eval172 proofs

Three weeks after second place, a fully agentic GPT-5.6 run is first on the Lean-Eval leaderboard, with 172 research-level problems solved. Every accepted proof is sorry-free and independently re-verified.

Three weeks ago this flow was second on Lean-Eval, behind a specialist prover. It is now first, with 172 research-level problems solved and machine-checked in Lean 4, using GPT-5.6 and a fully agentic workflow.

Lean-Eval · 18 August 2026, 13:28 UTC
  1. 1Seed Prover (ByteDance)156
  2. 2HumanfiaGPT-5.6, fully agentic172
  3. 3Aristotle (Harmonic)142
  4. 4GPT-5.6112
  5. 5Tau (caj.al)104
Accepted proofs per entry, counted from the board's own published data. The board opens in the order of 29 July and re-sorts into today's.

The model is one anybody can rent. What changed is the arrangement around it.

Three weeks, day by day ​

The board's published log dates every accepted proof. Read day by day, our first 130 proofs were accepted on 28 July, a second batch took us to 149 on 4 August, and a third to 167 on 16 August, past Seed Prover's 156 for the first time.

Lean-Eval · accepted proofs per entry · 28 July to 20 August

From #2 to #1

Swipe the plot sideways to see all of it →

What the loop does ​

Argue → formalize → two gates
PlanLeanerrorsclosed proofs1Comparator2AXLEACCEPTED

Swipe the diagram sideways to see all of it →

1/0 Argue first: a natural-language strategy, before any Lean.

Argue first, formalize second.

The flow writes a natural-language proof strategy before it writes any Lean, then turns that strategy into Lean statements. A formalization that starts from a plan fails in ways that are legible; one that starts from a blank file fails in ways that are not.

The compiler is the reviewer that never gets tired.

Lean's error output is fed back into the run and the code is refined against it, round after round, by HOA and RLCR Flow agents. Nothing about this is clever. Almost nobody does it for hundreds of rounds without a human losing patience.

Seed the hard ones with the easy ones.

For the problems that resist, the argument is first turned into a precise, stepwise informal plan, and the run is seeded with reusable Lean proofs from related problems that already closed. The agent spends its budget on the ideas that are actually missing.

What counts as solved ​

Two independent gates, run by two different agents, before anything is accepted:

  • the worker runs a comparator check on its own output; and
  • the reviewer re-verifies through the AXLE API, which has no access to how the proof was arrived at.

Nothing is accepted with a sorry placeholder or an unproved assumption standing in for a step.

A proof either has a term the Lean kernel accepts, or it does not exist.

That bar is the whole reason a leaderboard position in formal mathematics means anything at all.

HOA · the flows · the runtime

Read next

News#1 on Lean-Eval v1 first solvesLean-Eval froze its first 128-problem release and moved everything older to an archive. On v1 our GPT-5.6 flow has the most first solves, 30, and is one behind the leader on total; on the archive it has 170 of 171.Read it News30/30 IPhO, 6/6 IOI, 100/100 IBOIPhO 2026 theory 30 of 30, IOI 2026 six of six, IBO 2024 theory 100 of 100, IChO 2026 formalized 68 of 68, and quantum information theory 40 of 40. What each score is, who graded it, and what it does not show.Read it News672/672 on PutnamBench, joint #1GPT-5.6 in a worker–reviewer loop proved all 672 PutnamBench statements in Lean, at $44.50 a problem. The top of the official leaderboard is now a four-way tie, and what separates the four is cost.Read it