Skip to content
← All news

News3 min read

672/672 on PutnamBench, joint #1

GPT-5.6 in a worker–reviewer loop proved all 672 PutnamBench statements in Lean, at $44.50 a problem. The top of the official leaderboard is now a four-way tie, and what separates the four is cost.

In June this solver stood at 670 of 672. It has now closed the last two. Every one of PutnamBench's 672 formal statements has a Lean 4 proof that the PutnamBench team accepted, and the official leaderboard lists Humanfia at 672 (trishullab/PutnamBench#350, merged 1 September 2026).

Proved · Lean 4

672/672

Every formal statement in the benchmark, accepted by the PutnamBench team.

Average cost per problem

$44.50

GPT-5.6 at xhigh reasoning, as worker and as reviewer.

Entries at 672

4

The top line of the Lean column is a four-way tie.

How the top of the board filled up ​

PutnamBench went from tens of proofs to all of them in about a year. The board's best Lean score passed 600 in December 2025 and sat at Aleph Prover's 668 from January until the end of August, when two entries reached 672 a day apart. Ours was the second.

PutnamBench · Lean, with answers · best score on the board

From 47 to 672

Swipe the plot sideways to see all of it →

We are not alone up there. Four entries now report 672 in the Lean column with answers given, in the leaderboard data. Tied scores share a rank on the board below; hover or focus a row for its gap to the leader.

PutnamBench · Lean, with answers · top eight
  1. 1Aleph Prover (Logical Intelligence)added 26 Aug672
  2. 1Humanfiaadded 27 Aug · GPT-5.6 at xhigh672
  3. 1NEAR AI, with DeepSeek V4added 4 Sep672
  4. 1Forall (Astrio), with Claude Opus 5added 24 Sep672
  5. 5Mistral Prover (DeepSeek V4 Pro + Leanstral)added 4 Sep658
  6. 6Goedel-Architectadded 2 Jul642
  7. 7Leanstral 1.5added 19 Jul588
  8. 8Seed-Prover 1.5 (ByteDance)added Dec 2025581
From docs/results.json in trishullab/PutnamBench, read 5 October 2026.

What separates the four is cost ​

The four entries at 672 · reported cost per problem, in US dollars · sort by any column
Forall (Astrio), with Claude Opus 52026-09-24$3.52mean, actor-side only; median $2.49
NEAR AI, with DeepSeek V42026-09-04$0.17mean per problem; median $0.04
Humanfia2026-08-27$44.50average total per problem
Aleph Prover (Logical Intelligence)2026-08-26$74.00average per problem; maximum $1,468

The costs come from each team's own note on the leaderboard, and they are not measured the same way: one is a total, one is actor-side only. Read them as orders of magnitude, not as a ranking.Forall reports $2,364.39 actor-side in total and says reviewer and infrastructure costs are not included. NEAR AI's run used DeepSeek V4 Flash and V4 Pro. Read that way, the benchmark is saturated, and what separates the leaders now is the price of a proof, which is where we are not first. Two of the four got to 672 for a tenth of our cost or less.

The benchmark is saturated. What separates the leaders now is the price of a proof.

The board also has a separate "no answer" column, where the prover is not given the answer to problems that ask for one. There, Midas Prover reports 672 at an average of $6.17 per problem. Our run is in the column with answers, so the two are not the same test.

The loop: a worker, a reviewer, and Lean between them ​

The model that writes a proof never decides whether it is accepted. A worker writes the Lean file; Lean and Comparator check it; a separate reviewer, which cannot touch the file, asks the AXLE service for a verdict. Both are gpt-5.6-sol at xhigh, in separate processes with separate prompts and separate state.

Worker → Lean → reviewer
Workerno networkLean 4Reviewerno editsobjectionsACCEPTED

Swipe the diagram sideways to see all of it →

1/0 The worker writes a Lean proof, in a shell with no network access.

What counts as a proof ​

A problem counts only when the Lean file passes every gate in the solver's verification method. A candidate that fails any of them is kept for inspection, and never counted.

Seven gates · one way out
1234567COUNTED■ refused candidate■ real proof

Swipe the diagram sideways to see all of it →

1/0 Seven gates stand between a candidate and the count.

The statement is the benchmark's.

The problem's statement is taken from the pinned putnam_bench.jsonl by exact ID, and the workspace file must match it byte for byte.

And it stays the benchmark's.

A validator confirms the theorem and its docstring are unchanged. Weakened or restated theorems are rejected.

No escapes.

No sorry, admit, axiom or native_decide. The last is out because it trusts the compiler rather than the kernel.

It compiles.

Against the pinned Lean 4.27.0 and Mathlib.

Comparator agrees.

Comparator checks the statement, restricts the axioms to the standard three, and replays the proof through the Lean kernel. The worker cannot edit it or its configuration.

AXLE agrees.

A separate reviewer, which cannot edit the file, gets okay: true back from the AXLE verification API.

An audit after the fact.

Every pass is re-checked afterwards: hashes against the stored AXLE response, the statement validator again, and a clean rebuild.

The worker's shell has no network access, and no existing PutnamBench solution is ever mounted into either model's workspace.

Checking it ​

At the PutnamBench authors' request the full proof set is not public. The first 12 proofs are, along with the whole pipeline, and checking them through AXLE takes Python 3 and a network connection: the steps are in the repository.

humanfia/hoa-qed · HOA

Read next

News#1 on Lean-Eval v1 first solvesLean-Eval froze its first 128-problem release and moved everything older to an archive. On v1 our GPT-5.6 flow has the most first solves, 30, and is one behind the leader on total; on the archive it has 170 of 171.Read it News30/30 IPhO, 6/6 IOI, 100/100 IBOIPhO 2026 theory 30 of 30, IOI 2026 six of six, IBO 2024 theory 100 of 100, IChO 2026 formalized 68 of 68, and quantum information theory 40 of 40. What each score is, who graded it, and what it does not show.Read it News#1 on Lean-Eval: 172 proofsThree weeks after second place, a fully agentic GPT-5.6 run is first on the Lean-Eval leaderboard, with 172 research-level problems solved. Every accepted proof is sorry-free and independently re-verified.Read it