HMA · Humanize MLE Agents
Two coding agents take turns in one shared workspace, each turn in a fresh session. On 75 MLE-bench tasks in six hours, the pair won a medal on 78.2% of them.
78.2% any-medal rate.#1 of five. Opus 5 goes first, GPT-5.6-sol second, in six hours.
+8.0 pts over the mean of the two models alone.
01 How it works
≤ 5accepted
Or sooner, if the agent stops. Each agent uses its own native harness.
Newsession
The other agent takes over the same workspace.
6hours
The same budget each agent gets alone.
15minutes
No agent ever sees a private test score.
Crosses the handoff
Does not
No task-specific knowledge is added.
02 MLE-bench · 75 tasks
Share of the 75 tasks that won any medal.
Opus 5 → GPT-5.6-solMedal 78.2Gold 53.8Median+ 90.7Pctl 89.0 any medal vs the two alone +8.0
+8.0pts
Best pairing over the mean of its two models alone (+5.9 on mean percentile).
+6.4pts
The average gain across all 6 pairings tested. Use this one when comparing.
4.9pts
The gap between orders: 78.2% with Opus 5 first, 73.3% with GPT-5.6-sol first.
| Workflow | Budget | Any medal | Gold | Median+ | Mean percentile |
|---|---|---|---|---|---|
| HMA: Opus 5 ↔ GPT-5.6-sol | 6 h | 78.2 | 53.8 | 90.7 | 89.0 |
| HMA: GPT-5.6-sol ↔ Opus 5 | 6 h | 73.3 | 50.7 | 85.3 | 85.4 |
| Opus 5, native /goal | 6 h | 72.4 | 50.7 | 85.3 | 84.8 |
| HMA: GPT-5.6-sol ↔ DeepSeek V4.1 Flash | 6 h | 72.0 | 50.7 | 85.3 | 84.4 |
| HMA: GPT-5.6-sol ↔ Kimi K3 | 6 h | 70.7 | 50.7 | 85.3 | 83.9 |
| ScienceFlow | 24 h | 70.2 | — | — | — |
| GPT-5.6-sol, native /goal | 6 h | 68.0 | 47.6 | 81.3 | 81.3 |
| MLEvolve | 12 h | 65.3 | 34.7 | 76.0 | — |
03 Pairings
Each agent runs in its own native harness at max reasoning effort.
HMA pairing
Claude Opus 5thenGPT-5.6-sol
Claude Code → Codex
78.2% any medal
+8.0 pts over the mean of the two alone.
The other order: , 4.9 pts lower.
The 75 tasks, by complexity
04 Use it
Reproduce · humanfia/hma ↗
Baselines, HMA, ablations and external comparisons, with scripts for the tables and figures.
Use · a Humanize flow
flame_chasoid:fixed_interruptThe same method as a Humanize flow. It plays below.
Reruns still use the public answer keys, not the paper's corrected ones; the validation record lists what has been checked. More on Humanize.
The whole run, at rest. Step through it with the buttons, or drag the bar.
Each turn is a fresh session, closed the moment the evaluator has counted k accepted experiments — a number the agents are never told. At the deadline the agent that did not write the latest accepted candidate reviews them all blind, and may pick an older one.
Then round again: alternating, until only the review reserve is left.
It ends when the review settles the final candidate, a provider or the evaluator fails, or the budget runs out.
05 Before HMA
That earlier work, AgentKaggle, ran about ten agent workflows on live Kaggle leaderboards. It is not part of HMA's results: HMA reused only its data preparation, as its provenance file records.
Entered before the deadline, with an exact final rank.
Submitted after the close and scored against the frozen final board. An estimate, not a rank or a medal.
One square per completed competition.
Biohub · Cell Tracking · top % of the board
15 Aug · public board#8 of 2,378 · top 0.34%
Final · private board#188 of 3,947 · top 4.76%
agentkaggle/kaggle-results-audit marks each result official or late and links its evidence. The Team Radar leaderboard comes from the Kaggle API and is not audited. Per-entrant repositories are private because they hold competition data and credentials.
06 Timeline