Skip to content

Project · Open source · Apache-2.0 and CC-BY-4.0

HOAHumanfia Olympiad Agents

Agents solve olympiad and research mathematics, and Lean 4 checks the proofs. They also work on physics, chemistry, biology, programming and quantum information.

In mathematics a proof counts only if the Lean kernel accepts it. There is no partial credit.

SumOdd.lean✓ no goals
Accepted by the kernel

The record

Full marks on seven exams and the most first solves on Lean-Eval.#

One tile per exam or board, one cell per item, filled when done. The tag says who checked it; it is not always Lean.

PutnamBench · checked by Lean kernel

Joint first on the official leaderboard, at $44.50 a problem. 670 in June.

Written upAug 27Jun 26

No olympiad score here came from an official jury. Each write-up says who graded it.

PutnamBench

Every PutnamBench problem proved, level with three other entries.#

672 Putnam problems stated in Lean. We had 670 in June; the PutnamBench team accepted all 672 (#350). Three other entries reached 672 within a month. On cost per problem we are not first.

Official leaderboard · Lean, with answersA four-way tie at the top
  1. 1Humanfia · GPT-5.6 xhigh672
  2. 1Aleph Prover672
  3. 1NEAR AI · DeepSeek V4672
  4. 1Forall · Claude Opus 5672
  5. 5Mistral Prover658
  6. 6Goedel-Architect642
  7. 7Leanstral 1.5588
  8. 8Seed-Prover 1.5 (ByteDance)581

Each team's latest entry, from the leaderboard's data. Ties share a rank.

Reported cost per problem · log scaleOurs is the third-cheapest of the four at 672
  • Humanfia · GPT-5.6 xhigh$44.50 average
  • Aleph Prover$74 average · maximum $1,468
  • NEAR AI · DeepSeek V4$0.17 mean · median $0.04
  • Forall · Claude Opus 5$3.52 mean, actor side only

Each team's own figure, measured differently, so compare orders of magnitude. Two of the four reached 672 for a tenth of our cost or less.

Lean-Eval

Research mathematics: the most first solves.#

Lean-Eval is theorems from research papers, stated in Lean. Its frozen v1 has 128 problems and ranks entries three ways.

The first accepted proof of a problem.
  1. Humanfia · GPT-5.630Total 89Unique 0
  2. Axiom Prover23Total 90Unique 0
  3. NEAR AI · DeepSeek V412Total 88Unique 0
  4. max gp · GPT-64Total 92Unique 2

First on the board: the first accepted proof on 30 of the 92 solved problems.

LeanEval v1
128 problems, frozen 20 August 2026; 92 solved by anyone
Archive
170 / 171 accepted; Axiom Prover has all 171
Both scopes
259 distinct problems accepted, two behind Axiom Prover

Board data from 23 October 2026. The combined 259 is our count; the board has no combined column.

IMO 2026

All six IMO 2026 problems proved in Lean, on each of two backends.#

Two workers on two backends each proved all six in Lean. Times are API minutes, the time spent inside model calls.

  1. Q138.187.1241.6× slower
  2. Q2100.4224.33603.6× faster
  3. Q3187.1343.78694.6× faster
  4. Q458.775.6391.5× slower
  5. Q546.591.9651.4× faster
  6. Q666.9212.41392.1× faster
AxiomProver's times are as it reported them. Kimi-K3's include its Codex reviewer. GPT-5.6 is slower than AxiomProver on Q1 and Q4, the two easiest, and 4.6× faster on Q3, the hardest.

What counts as solved

A proof counts only after five checks the prover does not control.#

A problem counts only when its Lean file passes all five gates, none of them run by the model that wrote the proof. Pick a file to see where it stops.

  1. 1StatementByte-identical to the pinned benchmark statement.
  2. 2No sorryNo sorry, admit, axiom or native_decide.
  3. 3CompilesBuilds on the pinned Lean 4.27.0 and Mathlib.
  4. 4ComparatorChecks the statement and replays the proof in the kernel.
  5. 5AXLE reviewA reviewer that cannot edit the file gets okay: true from AXLE.
  6. CountedOnly then does the problem count.

honest.lean Passes all five gates, so it counts.

Semantic review

Does the theorem say what the problem said?

A proof Lean accepts can still prove the wrong theorem. This is checked and reported separately.

Around the gates

The worker has no network access, and neither model's workspace holds an existing PutnamBench solution. Details: the verification method, Comparator and AXLE.

With and without a flow

The same model proves 3 of 50 Putnam problems through the API and 50 inside a flow.#

The models are publicly available. The flow decides who works, who reviews and what carries over between attempts.

Same model · three levels of scaffolding · solved of 50
01020304050Model APICoding CLIHumanize flow50 GPT-5.6-sol25 GLM-5.313 DeepSeek V4 Pro47 Kimi-K3
From the KDA² ablation, September 2026. Physics Cup gains less: not every benchmark is limited by the loop. The July comparison adds SuperChem and an HLE subset.

IChO 2026 · theory subquestions accepted

Same model, with and without the review loop.

  • GPT-5.6 Solnative /goal 32review loop 68 / 68
  • Kimi-K3native /goal 31review loop 68 / 68

About half accepted as a plain Codex /goal; all 68 with formalization and proof review.

Where the code is

Every proof, flow and README is in one public repository.#

All of HOA is humanfia/hoa: one directory per competition or library, each with a README.

Results, as they came in

Results by date, June to October 2026.#

Each card links to its write-up.