OpenBench – A benchmark for comparing coding-agent harnesses

OpenBench is a benchmark aimed at comparing coding-agent harnesses. It was shared by mattlam_ on X at https://twitter.com/mattlam_/status/2079605387121049605 and surfaced on Hacker News under the AI Agents track at https://news.ycombinator.com/item?id=49000462, where it had 1 point and 0 comments at the time of the summary.

The focus is harness comparison rather than model leaderboards alone. A harness is the layer around a coding agent: how tools are exposed, how context and files are fed in, how runs are sandboxed, scored, and repeated. OpenBench treats that stack as the unit under test so differences in scaffolding show up next to differences in the model.

For engineers building agents, that matters because product quality often lives in the glue: tool schemas, retry policy, workspace layout, and evaluation hooks. A harness-level benchmark lets a team ask which runner setup produces more reliable coding outcomes under the same task set, instead of only swapping model endpoints and guessing why results moved.

The market already has many model and coding benchmarks; fewer public efforts isolate the agent harness as the variable. OpenBench sits in that thinner slice: useful for teams comparing open-source runners, internal scaffolds, and productized agent platforms on equal tasks rather than on marketing claims.

What to watch next is whether the suite, tasks, and scoring method get documented enough that independent runs can be reproduced, and whether early HN silence turns into actual adoption by builders who publish side-by-side harness results against the same OpenBench criteria.