gem5's build system was ported from SCons to CMake in full by one engineer directing agents for weeks. The result is gem5/gem5#2969: 567 files changed across 133 commits. CMake + Ninja becomes the build, and a Bazel build runs alongside it without touching a single source file.
Files changed
567All of SCons out, CMake and Bazel in, and every doc, CI job and script that named the old build.
Commits
133One engineer, with agents doing the typing, over several weeks.
Lines added · 16.5k removed
+26.5kMostly the new builds: 10,003 lines of CMake and 14,244 of Bazel. 15,794 of the removed lines were SCons.
This is the oldest result on this page and, for what we were trying to learn, one of the most useful. A build-system migration is the least glamorous long-horizon task there is, and it has almost none of the properties that make a benchmark flattering.
The scale
Most of the change is a swap: 193 SCons files out, 147 CMake files in. Then there is the long tail of everything that named the old build, such as a binary path in a config, a command in a Dockerfile or a step in a CI job. None of it is hard, and all of it has to be found.
- Count
Two builds, one source tree
The port had one hard constraint, which is that the source tree does not move. gem5 generates a large part of its own C++ (the ISA parser, the SLICC protocol compiler, the SimObject wrappers, the embedded Python), and every one of those pipelines had to come across unchanged and keep running on the same Python tools.
Swipe the diagram sideways to see all of it →
1/0 One source tree, built by SCons: 193 SConstruct, SConscript and site_scons files.
The same simulator, twice.
The binary from each build was checked against the other's, object by object.
Green under both.
testlib-quick passes under CMake, and every Bazel unit test passes. Both binaries run a hello-world SE simulation on x86 and on Arm.
Everything that said scons.
CI workflows, Dockerfiles, example configs and utility scripts, down to binary paths such as build/X86/gem5.opt becoming build/ALL/gem5.
Why this task taught us the most
There is no clever insight to have, and no moment where a good idea collapses the search. There is a very large number of mechanical changes, each of them easy. The build either works or does not, and there is a long stretch in the middle where it does not work for a reason that has nothing to do with the change you just made.
A session that starts fresh every turn falls apart on work of this shape, and a run that keeps everything in context drowns in it.
An agent left unsupervised will eventually declare victory on a build that compiles only part of the tree. What held this one together was the loop around the agents: a plan agreed up front, then rounds of building under an independent review. That loop was RLCR Flow, then a Claude Code plugin. It now lives on as the humanize1 flow, the same plan-then-review loop run by Humanize. This task is why we started caring about the loop rather than the model, and it is the kind of task FlowBench is being assembled from.
The pull request is open upstream as a request for comments, and the gem5 maintainers decide what happens to it next.The pull request's description, written when it was opened, says 563 files and about 24k insertions. The counts on this page are from its file list as it stands today, which is a little larger.
The pull request, gem5/gem5#2969 · The humanize1 flow · FlowBench