Skip to content

HMA · Humanize MLE Agents

Taking turns beats either agent alone

Two coding agents take turns in one shared workspace, each turn in a fresh session. On 75 MLE-bench tasks in six hours, the pair won a medal on 78.2% of them.

MLE-bench · any medal · 75 tasksself-reported ↗

78.2% any-medal rate.#1 of five. Opus 5 goes first, GPT-5.6-sol second, in six hours.

  1. 1Opus 5 ↔ GPT-5.6-sol 78.2
  2. 2Opus 5 native /goal72.4
  3. 3ScienceFlow 24 h70.2
  4. 4GPT-5.6-sol native /goal68.0
  5. 5MLEvolve 12 h65.3

+8.0 pts over the mean of the two models alone.

One task · 6 h · schematic
6h 00m21 accepted · session 5
Turn lengths are illustrative.

01 How it works

Each turn hands over the workspace but not the context

  1. 01

    ≤ 5accepted

    A turn ends after five accepted submissions

    Or sooner, if the agent stops. Each agent uses its own native harness.

  2. 02

    Newsession

    Each turn opens a fresh session

    The other agent takes over the same workspace.

  3. 03

    6hours

    Both agents share one six-hour budget

    The same budget each agent gets alone.

  4. 04

    15minutes

    A final review picks one accepted candidate

    No agent ever sees a private test score.

Crosses the handoff

  • Code
  • Models
  • Evaluated results
  • Candidates

Does not

  • The last agent's context
  • Any private test score

Why it works: fresh context and a second model

Context renewal
A fresh session restarts the improvement curve that a long session flattens.
Complementary capabilities
The second model searches differently from the first.

No task-specific knowledge is added.

Schematic, not data
agent A turnagent B turnone agent, one sessiontime →

02 MLE-bench · 75 tasks

The best pairing beats harnesses given up to four times as long

Share of the 75 tasks that won any medal.

  1. 1Opus 5 → GPT-5.6-solHMA · 6 h78.2
  2. 2GPT-5.6-sol → Opus 5HMA · 6 h73.3
  3. 3Opus 5native /goal · 6 h72.4
  4. 4GPT-5.6-sol → DeepSeek V4.1 FlashHMA · 6 h72.0
  5. 5GPT-5.6-sol → Kimi K3HMA · 6 h70.7
  6. 6ScienceFlowexternal · 24 h70.2
  7. 7GPT-5.6-solnative /goal · 6 h68.0
  8. 8MLEvolveexternal · 12 h65.3

Opus 5 → GPT-5.6-solMedal 78.2Gold 53.8Median+ 90.7Pctl 89.0 any medal vs the two alone +8.0

+8.0pts

Best pairing over the mean of its two models alone (+5.9 on mean percentile).

+6.4pts

The average gain across all 6 pairings tested. Use this one when comparing.

4.9pts

The gap between orders: 78.2% with Opus 5 first, 73.3% with GPT-5.6-sol first.

The full table
WorkflowBudgetAny medalGoldMedian+Mean percentile
HMA: Opus 5 ↔ GPT-5.6-sol6 h78.253.890.789.0
HMA: GPT-5.6-sol ↔ Opus 56 h73.350.785.385.4
Opus 5, native /goal6 h72.450.785.384.8
HMA: GPT-5.6-sol ↔ DeepSeek V4.1 Flash6 h72.050.785.384.4
HMA: GPT-5.6-sol ↔ Kimi K36 h70.750.785.383.9
ScienceFlow24 h70.2———
GPT-5.6-sol, native /goal6 h68.047.681.381.3
MLEvolve12 h65.334.776.0—

03 Pairings

Of 6 pairings tested, Opus 5 first and GPT-5.6-sol second does best

Each agent runs in its own native harness at max reasoning effort.

Opus 5Claude CodeGPT-5.6CodexGLM-5.3Claude CodeDS FlashDeepSeek HarnessKimi K3Kimi Code
Opus 5Claude Code
GPT-5.6Codex
GLM-5.3Claude Code
DS FlashDeepSeek Harness
Kimi K3Kimi Code

HMA pairing

Claude Opus 5thenGPT-5.6-sol

Claude Code → Codex

78.2% any medal

  • Any medal78.2
  • Gold53.8
  • Median+90.7
  • Mean percentile89.0

+8.0 pts over the mean of the two alone.

The other order: , 4.9 pts lower.

The 75 tasks, by complexity

1 × A10
NVIDIA GPU per task
30
vCPUs per task
220 GiB
RAM per task
27
configurations in the full plan
3,397
task-runs in the full plan
16
tasks in the external-harness subset

04 Use it

Rerun the paper's experiments, or run HMA as a Humanize flow

Reruns still use the public answer keys, not the paper's corrected ones; the validation record lists what has been checked. More on Humanize.

flame_chasoid:fixed_interruptsimulated

The whole run, at rest. Step through it with the buttons, or drag the bar.

Each turn is a fresh session, closed the moment the evaluator has counted k accepted experiments — a number the agents are never told. At the deadline the agent that did not write the latest accepted candidate reviews them all blind, and may pick an older one.

The run, turn by turn
  1. first_chaser — experiments; a session opened for this turn; hands evaluator in the tree: 5 accepted
  2. evaluator — k = 5: closed; no turn of a model; hands second_chaser in the tree: the workspace
  3. second_chaser — experiments; a session opened for this turn; hands evaluator in the tree: 5 accepted
  4. evaluator — k = 5: closed; no turn of a model; hands first_chaser in the tree: the workspace
  5. first_chaser — experiments; a session opened for this turn; hands evaluator in the tree: 3 accepted
  6. evaluator — stops counting; no turn of a model; hands second_chaser in the tree: 13 candidates
  7. second_chaser — blind review; a session opened for this turn; hands the finish as words: picks an older one

Then round again: alternating, until only the review reserve is left.

It ends when the review settles the final candidate, a provider or the evaluator fails, or the budget runs out.

05 Before HMA

HMA grew out of our work on live Kaggle competitions

That earlier work, AgentKaggle, ran about ten agent workflows on live Kaggle leaderboards. It is not part of HMA's results: HMA reused only its data preparation, as its provenance file records.

Official finishes and late estimates are never counted together

Official

Entered before the deadline, with an exact final rank.

Late

Submitted after the close and scored against the frozen final board. An estimate, not a rank or a medal.

It ended with 3 official top-5% finishes in 39 competitions

One square per completed competition.

Biohub · Cell Tracking · top % of the board

top 5%

15 Aug · public board#8 of 2,378 · top 0.34%

Final · private board#188 of 3,947 · top 4.76%

The public rank was provisional. The final private rank is the official result.
Official final ranks in the top 5%
  1. 1Predicting Student Health Risktop 2.00%
  2. 2ROGII Wellbore Geologytop 2.24%
  3. 3Biohub · Cell Tracking188th of 3,947top 4.76%
Top % of the final private board. Lower is better.

Every Kaggle number here comes from one audit

agentkaggle/kaggle-results-audit marks each result official or late and links its evidence. The Team Radar leaderboard comes from the Kaggle API and is not audited. Per-entrant repositories are private because they hold competition data and credentials.