Skip to content
← All news

News2 min read

78.2% MLE-bench medal rate in 6 hours

Two coding agents taking turns over one workspace, each starting fresh, beat both of them working alone on 75 MLE-bench tasks: 78.2% any-medal against 72.4% and 68.0%, in the same six hours. Self-reported, and not on the official leaderboard.

Take two strong coding agents. Give each the same machine-learning task and six hours, and Claude Opus 5 wins a medal on 72.4% of MLE-bench's 75 Kaggle tasks while GPT-5.6-sol wins one on 68.0%. Now give them one six-hour budget and make them take turns in the same workspace. Together they medal on 78.2%.

That is Humanize MLE Agents (HMA), and the number comes from its results table.

Best pairing

+8.0 pts

Opus 5 ⇄ GPT-5.6-sol, over the mean of the two alone (70.2%), in any-medal rate.

All six pairings

+6.4 pts

The average gain over each pairing’s own single-agent mean. The conservative figure.

Mean percentile

+5.9 pts

Where the best pairing lands on each task’s leaderboard, against the mean of its two agents alone.

Taking turns, with a clean slate ​

The arrangement is simple, and almost all of it is about what is not carried over.

One task · one workspace · six hours
0 h6 hshared workspacecode · models · candidatesOpus 5GPT-5.6Opus 5GPT-5.6last 15 min: pick one

Swipe the diagram sideways to see all of it →

1/0 One shared workspace, and one six-hour budget for the task.

Turn lengths vary by task, so the ones drawn here are illustrative. The cap and the 15 minutes are not.
  1. One agent works the task in its own native harness and submits candidate solutions.
  2. After at most five accepted submissions, or when it finishes on its own, it stops. The other agent starts a fresh session in the same workspace: the same code, models and candidates, but none of the first agent's context.
  3. They alternate until the six hours are nearly up. The last 15 minutes go to picking one of the candidates already accepted. Private test scores are never shown to either agent.

The repository credits two effects at once. The first is context renewal: a long-running agent improves more and more slowly, and a fresh session restarts that curve. The second is complementary capabilities: the second model searches differently from the first. The handoff keeps everything the first agent built and drops only its train of thought.

Against each agent alone ​

MLE-bench · 75 tasks · 6 hours each
68.0%−4.4%
72.4%
78.2%+5.8%
Any medal
47.6%−3.1%
50.7%
53.8%+3.1%
Gold
81.3%−4.0%
85.3%
90.7%+5.4%
Above median
81.3%−3.5%
84.8%
89.0%+4.2%
Mean percentile
All values are percentages from the HMA repository. Choose a baseline to read the gain over it.

The same table lists two external systems with longer budgets: ScienceFlow reaches 70.2% in 24 hours, and MLEvolve 65.3% in 12. Every HMA pairing uses six.

Any-medal rate against time budget
  • GPT-5.6-sol alone: Time budget 6 h, Any medal 68.0%
  • GPT-5.6-sol ⇄ Kimi K3: Time budget 6 h, Any medal 70.7%
  • GPT-5.6-sol ⇄ DeepSeek V4.1 Flash: Time budget 6 h, Any medal 72.0%
  • Opus 5 alone: Time budget 6 h, Any medal 72.4%
  • GPT-5.6-sol ⇄ Opus 5: Time budget 6 h, Any medal 73.3%
  • Opus 5 ⇄ GPT-5.6-sol: Time budget 6 h, Any medal 78.2%
  • MLEvolve: Time budget 12 h, Any medal 65.3%
  • ScienceFlow: Time budget 24 h, Any medal 70.2%

Every workflow in the HMA results table · all values %
HMA: Opus 5 ⇄ GPT-5.6-sol678.253.890.789.0
HMA: GPT-5.6-sol ⇄ Opus 5673.350.785.385.4
Claude Opus 5, native /goal672.450.785.384.8
HMA: GPT-5.6-sol ⇄ DeepSeek V4.1 Flash672.050.785.384.4
HMA: GPT-5.6-sol ⇄ Kimi K3670.750.785.383.9
ScienceFlow2470.2———
GPT-5.6-sol, native /goal668.047.681.381.3
MLEvolve1265.334.776.0—

What to be careful with ​

  • Order matters a lot. The same two models score 78.2% with Opus going first and 73.3% with GPT-5.6-sol going first. Choosing the best order after the fact flatters the headline. The average over pairings, 6.4 points, is the more conservative figure.
  • This is self-reported, and it is not on the official leaderboard. The MLE-bench leaderboard has not reviewed these runs, and it has not taken new submissions since April. For scale only, its highest "All" entry today is 64.44%, with a 24-hour budget and an older model. Different models, budgets and repeat counts make that a different experiment, not a lower bar.
  • The paper is not out yet. The arXiv link will be added with the paper release. Until then, the full ablations exist only in the repository's experiment map.

The whole suite reruns from the repository: the baselines, HMA, and the cap and starting-order ablations. To use the same alternation on your own work, run it as a flow: fixed-interrupt flame chase.

humanfia/hma · HMA

Read next

NewsBiohub final: #188 of 3,947, top 5%In August an agent workflow sat 8th on the Biohub cell-tracking public leaderboard. The final private rank is 188th of 3,947, the top 4.76%: lower, but an official finish. Plus the Kaggle tally when only final ranks count.Read it NewsTwo Kaggle finishes in the top 3%Ten agent workflows on Kaggle, with every result sorted by what kind of rank it is. Two closed with official final ranks in the top 3%. Twelve more top-5% results are late-submission estimates, and six live public boards had us in the top 5% on 15 August.Read it News#1 on Lean-Eval v1 first solvesLean-Eval froze its first 128-problem release and moved everything older to an archive. On v1 our GPT-5.6 flow has the most first solves, 30, and is one behind the leader on total; on the archive it has 170 of 171.Read it