In June this solver stood at 670 of 672. It has now closed the last two. Every one of PutnamBench's 672 formal statements has a Lean 4 proof that the PutnamBench team accepted, and the official leaderboard lists Humanfia at 672 (trishullab/PutnamBench#350, merged 1 September 2026).
Proved · Lean 4
672/672Every formal statement in the benchmark, accepted by the PutnamBench team.
Average cost per problem
$44.50GPT-5.6 at xhigh reasoning, as worker and as reviewer.
Entries at 672
4The top line of the Lean column is a four-way tie.
How the top of the board filled up
PutnamBench went from tens of proofs to all of them in about a year. The board's best Lean score passed 600 in December 2025 and sat at Aleph Prover's 668 from January until the end of August, when two entries reached 672 a day apart. Ours was the second.
From 47 to 672
Swipe the plot sideways to see all of it →
We are not alone up there. Four entries now report 672 in the Lean column with answers given, in the leaderboard data. Tied scores share a rank on the board below; hover or focus a row for its gap to the leader.
- 1Aleph Prover (Logical Intelligence)added 26 Aug672
- 1Humanfiaadded 27 Aug · GPT-5.6 at xhigh672
- 1NEAR AI, with DeepSeek V4added 4 Sep672
- 1Forall (Astrio), with Claude Opus 5added 24 Sep672
- 5Mistral Prover (DeepSeek V4 Pro + Leanstral)added 4 Sep658
- 6Goedel-Architectadded 2 Jul642
- 7Leanstral 1.5added 19 Jul588
- 8Seed-Prover 1.5 (ByteDance)added Dec 2025581
What separates the four is cost
| Forall (Astrio), with Claude Opus 5 | 2026-09-24 | $3.52 | mean, actor-side only; median $2.49 |
|---|---|---|---|
| NEAR AI, with DeepSeek V4 | 2026-09-04 | $0.17 | mean per problem; median $0.04 |
| Humanfia | 2026-08-27 | $44.50 | average total per problem |
| Aleph Prover (Logical Intelligence) | 2026-08-26 | $74.00 | average per problem; maximum $1,468 |
The costs come from each team's own note on the leaderboard, and they are not measured the same way: one is a total, one is actor-side only. Read them as orders of magnitude, not as a ranking.Forall reports $2,364.39 actor-side in total and says reviewer and infrastructure costs are not included. NEAR AI's run used DeepSeek V4 Flash and V4 Pro. Read that way, the benchmark is saturated, and what separates the leaders now is the price of a proof, which is where we are not first. Two of the four got to 672 for a tenth of our cost or less.
The benchmark is saturated. What separates the leaders now is the price of a proof.
The board also has a separate "no answer" column, where the prover is not given the answer to problems that ask for one. There, Midas Prover reports 672 at an average of $6.17 per problem. Our run is in the column with answers, so the two are not the same test.
The loop: a worker, a reviewer, and Lean between them
The model that writes a proof never decides whether it is accepted. A worker writes the Lean file; Lean and Comparator check it; a separate reviewer, which cannot touch the file, asks the AXLE service for a verdict. Both are gpt-5.6-sol at xhigh, in separate processes with separate prompts and separate state.
Swipe the diagram sideways to see all of it →
1/0 The worker writes a Lean proof, in a shell with no network access.
What counts as a proof
A problem counts only when the Lean file passes every gate in the solver's verification method. A candidate that fails any of them is kept for inspection, and never counted.
Swipe the diagram sideways to see all of it →
1/0 Seven gates stand between a candidate and the count.
The statement is the benchmark's.
The problem's statement is taken from the pinned putnam_bench.jsonl by exact ID, and the workspace file must match it byte for byte.
And it stays the benchmark's.
A validator confirms the theorem and its docstring are unchanged. Weakened or restated theorems are rejected.
No escapes.
No sorry, admit, axiom or native_decide. The last is out because it trusts the compiler rather than the kernel.
It compiles.
Against the pinned Lean 4.27.0 and Mathlib.
Comparator agrees.
Comparator checks the statement, restricts the axioms to the standard three, and replays the proof through the Lean kernel. The worker cannot edit it or its configuration.
AXLE agrees.
A separate reviewer, which cannot edit the file, gets okay: true back from the AXLE verification API.
An audit after the fact.
Every pass is re-checked afterwards: hashes against the stored AXLE response, the statement validator again, and a clean rebuild.
The worker's shell has no network access, and no existing PutnamBench solution is ever mounted into either model's workspace.
Checking it
At the PutnamBench authors' request the full proof set is not public. The first 12 proofs are, along with the whole pipeline, and checking them through AXLE takes Python 3 and a network connection: the steps are in the repository.