Skip to content
← All news

News1 min read

670/672 on PutnamBench

One Ralph loop, with Codex as both worker and reviewer, closed 670 of PutnamBench's 672 formal statements, every problem of Putnam 2025 among them. The best on the leaderboard that day was 668.

This has since moved

The solver later closed the last two: 672 of 672, joint first on the official leaderboard, every proof through Lean 4, Comparator and AXLE.

PutnamBench is 672 formal statements from the Putnam competition, 1962 to 2025. A Ralph loop running Codex as both worker and reviewer closed 670 of them, 99.7%, including every problem in Putnam 2025.

PutnamBench · Lean, with answers · 26 June 2026
  1. 1Humanfiaour report · 99.7%670
  2. 2Aleph Prover (Logical Intelligence)added Jan 2026 · 99.4%668
  3. 3Goedel-Architectadded Jun 2026 · 88.8%597
  4. 4Seed-Prover 1.5 (ByteDance)added Dec 2025 · 86.5%581
  5. 5Hilbertadded Oct 2025 · 68.8%462
  6. 6AxProverBase (Axiomatic AI)added Apr 2026 · 54.3%365
Every entry but ours is from the leaderboard's own data, as it stood on 26 June. Our 670 is our own report and was never a board entry; the board later listed us at 672.

The two that are not closed are not closed. They are drawn as such everywhere we show this.

Closed

670/672

One loop, one model family, every proof checked by Lean.

Of the benchmark

99.7%

Against 99.4% for Aleph Prover, the best on the board that day.

Still open

2

Not closed, and not counted.

The loop ​

A Ralph loop is the plainest loop there is: give an agent the same goal again and again, with the last attempt's feedback, until the goal is met or the budget runs out. Here the goal is one PutnamBench statement in its own workspace, the feedback is Lean's, and a second Codex process judges what compiles.

One problem · one Ralph loop
WorkerLeanReviewerturn after turncampaign 2

Swipe the diagram sideways to see all of it →

1/0 One problem, one workspace: a Codex worker writes a Lean proof.

Where the turns go ​

More than 90% of problems are solved within ten turns.The turn counts are from our run logs and are not published; read them as Humanfia-reported. The solver's README gives the configuration: GPT-5.5 at xhigh reasoning, a 50-turn cap per problem in each campaign, and later campaigns that retry only what is still open. For most of the benchmark the loop is not doing anything exotic: it writes a proof, the compiler objects, it fixes it, and it is done well inside the budget.

Turns per problem · one campaign
  1. 110 turns over 90% of problems solved by here
  2. 250 turns the cap per problem, per campaign
At this turnmost of the benchmark is closed

The tail is where the argument is. For the harder problems, extra turns raise the solve rate measurably: the same worker, given more rounds against the same feedback, closes problems it did not close in ten.

The model was capable of the proof the whole time. What was missing was the arrangement that let it keep going without wandering.

That is the thesis of this site, stated as an experiment.

Checking it ​

The official leaderboard is maintained by the PutnamBench team. A preview set of the proof files is open for review, and the full solving pipeline is public.

humanfia/hoa-qed/putnambench · HOA

Read next

News#1 on Lean-Eval v1 first solvesLean-Eval froze its first 128-problem release and moved everything older to an archive. On v1 our GPT-5.6 flow has the most first solves, 30, and is one behind the leader on total; on the archive it has 170 of 171.Read it News30/30 IPhO, 6/6 IOI, 100/100 IBOIPhO 2026 theory 30 of 30, IOI 2026 six of six, IBO 2024 theory 100 of 100, IChO 2026 formalized 68 of 68, and quantum information theory 40 of 40. What each score is, who graded it, and what it does not show.Read it News672/672 on PutnamBench, joint #1GPT-5.6 in a worker–reviewer loop proved all 672 PutnamBench statements in Lean, at $44.50 a problem. The top of the official leaderboard is now a four-way tie, and what separates the four is cost.Read it