Three weeks ago this flow was second on Lean-Eval, behind a specialist prover. It is now first, with 172 research-level problems solved and machine-checked in Lean 4, using GPT-5.6 and a fully agentic workflow.
- 1Seed Prover (ByteDance)156
- 2HumanfiaGPT-5.6, fully agentic172
- 3Aristotle (Harmonic)142
- 4GPT-5.6112
- 5Tau (caj.al)104
The model is one anybody can rent. What changed is the arrangement around it.
Three weeks, day by day
The board's published log dates every accepted proof. Read day by day, our first 130 proofs were accepted on 28 July, a second batch took us to 149 on 4 August, and a third to 167 on 16 August, past Seed Prover's 156 for the first time.
From #2 to #1
Swipe the plot sideways to see all of it →
What the loop does
Swipe the diagram sideways to see all of it →
1/0 Argue first: a natural-language strategy, before any Lean.
Argue first, formalize second.
The flow writes a natural-language proof strategy before it writes any Lean, then turns that strategy into Lean statements. A formalization that starts from a plan fails in ways that are legible; one that starts from a blank file fails in ways that are not.
The compiler is the reviewer that never gets tired.
Lean's error output is fed back into the run and the code is refined against it, round after round, by HOA and RLCR Flow agents. Nothing about this is clever. Almost nobody does it for hundreds of rounds without a human losing patience.
Seed the hard ones with the easy ones.
For the problems that resist, the argument is first turned into a precise, stepwise informal plan, and the run is seeded with reusable Lean proofs from related problems that already closed. The agent spends its budget on the ideas that are actually missing.
What counts as solved
Two independent gates, run by two different agents, before anything is accepted:
- the worker runs a comparator check on its own output; and
- the reviewer re-verifies through the AXLE API, which has no access to how the proof was arrived at.
Nothing is accepted with a sorry placeholder or an unproved assumption standing in for a step.
A proof either has a term the Lean kernel accepts, or it does not exist.
That bar is the whole reason a leaderboard position in formal mathematics means anything at all.
HOA · the flows · the runtime