This has since moved
Three weeks later the same loop took first place with 172 problems.
Running on gpt-5.6-sol at max effort, Humanize is second on the Lean-Eval leaderboard. By our count today it has solved 149 of 219 problems. First is ByteDance's Seed Prover, at 155.Those two counts are ours, from the day. The board's published log dates every proof by when it was accepted, and by that log the standings at the end of 29 July were 144 for Seed Prover and 130 for us; it records our 149th on 4 August. The order is the same either way.
- 1Seed Prover (ByteDance)144
- 2HumanfiaGPT-5.6 sol, max effort130
- 3Aristotle (Harmonic)124
- 4GPT-5.6112
- 5Tau (caj.al)104
A specialist and a generalist
Six problems by our count, fourteen by the board's log: either way it is not a rout, and the gap is worth naming precisely because of what is on each side of it. Seed Prover is a system built for this: a prover, trained and tuned for formal mathematics. What is second is a general-purpose model driving a general-purpose agent loop, with no component anywhere in it that knows Lean specifically.
Swipe the diagram sideways to see all of it →
1/0 Seed Prover is a prover: built, trained and tuned for Lean.
The loop is the same one that runs kernel work and Kaggle competitions.
A specialist should beat a generalist on the specialist's benchmark. That it does so by so little is the interesting number.
That is why we thought the gap was closable.