AI Coding Agent Security Benchmark news update

Endor Labs launched the Agent Security League, an open benchmark tracking functional and security correctness across commercial AI coding agents. As detailed in endorlabs.com's report, researchers evaluated 33 harness and model combinations on real-world programming tasks. While functional capabilities reached 87.2 percent, security correctness peaked at only 37.4 percent across all tested configurations.

This report details the benchmark results, evaluation methodology, and comparative performance across every tested developer toolchain. It examines why generated code regularly introduces vulnerabilities, how anti-cheating protections function, and which harness configurations achieve the highest reliability. This analysis is written for software engineers, application security teams, and engineering leaders evaluating autonomous tools like Claude Code, Cursor, Codex, and SWE-Agent for production repositories.

AI Coding Agent Security Benchmark news update: what actually changed

The Agent Security League leaderboard ranks 33 distinct agent and model configurations evaluated between February and October 2026. Across all tested pairings, Claude Code combined with Claude Fable 5.1 took first place on September 4, 2026, scoring 87.2 percent functional correctness and 37.4 percent security correctness. Codex paired with GPT-6 Astra achieved second place on September 9, 2026, with 82.1 percent functional and 34.6 percent security correctness. Codex with GPT-6.1 Sol took third on October 1, 2026, recording 77.7 percent functional and 34.1 percent security scores.

Claude Code running Claude Opus 5.5 reached fourth place on September 24, 2026, with 68.7 percent functional and 33.5 percent security correctness, while Claude Code with Claude Opus 5 took fifth on July 31, 2026, at 73.7 percent functional and 32.4 percent security. Conversely, Cursor with GPT-5.5 reached 84.9 percent functional correctness on April 25, 2026, but finished ninth with 24.0 percent security correctness. At the bottom, SWE-Agent with Gemini 2.5 Pro finished thirty-third on February 19, 2026, with 20.7 percent functional and 4.5 percent security correctness.

AI Coding Agent Security Benchmark news update: how it works

The Agent Security League evaluation methodology builds directly upon SusVibes, a foundational peer-reviewed research benchmark created at Carnegie Mellon University. Testing comprises 200 software engineering tasks extracted from 108 open-source Python repositories, covering 77 Common Weakness Enumeration vulnerability classifications. Performance is assessed through two separate automated metrics: functional correctness, termed FuncPass, and security correctness, termed SecPass. Under the FuncPass standard, generated code must successfully pass a repository's full functional test suite to confirm operational execution. Under SecPass, that code must also pass security test suites designed to detect vulnerability introductions.

The evaluation pipeline incorporates automated anti-cheating safeguards, including prompt hardening, workspace sanitization, and automated cheating detection. These controls prevent autonomous coding agents from bypassing harness boundaries, tampering with inspection scripts, or suppressing assertion failures during execution. The benchmark tracks four autonomous harnesses: Claude Code, Codex, Cursor, and SWE-Agent. The testing matrix pairs these environments with 25 frontier model releases, including Claude Fable 5, Claude Sonnet 4.6, Claude Opus 4.8, GPT-5.4, GPT 5.6 Sol, GPT-6 Sol, Gemini 3 Pro, Gemini 3.5 Flash, Gemini 4 Argon, and Composer 2.5.

AI Coding Agent Security Benchmark news update: why it matters now

The benchmark results expose a profound structural deficit in how autonomous coding agents construct software. Even the top-performing combination, Claude Code with Claude Fable 5.1, failed security validation on 62.6 percent of its functionally verified solutions across 77 Common Weakness Enumeration classes. Across the 33 benchmarked pairings, most configurations achieved security correctness scores below 25 percent. This pattern proves that optimizing models solely to pass functional unit tests leads to vulnerable code generation, leaving critical vulnerabilities hidden within syntactically correct code that passes normal continuous integration checks.

The timeline of results reveals steady progress across 2026 releases, yet absolute security levels remain inadequate for unmonitored production pipelines. In February 2026, early pairings like Claude Code with Claude Sonnet 4 recorded only 6.1 percent security correctness, while Claude Code with Gemini 3 Pro achieved 9.5 percent. By October 2026, Cursor with Gemini 4 Argon reached 29.0 percent security, and Codex with GPT-6.1 Sol achieved 34.1 percent. Although security scores improved substantially throughout the year, commercial agents still introduce vulnerabilities in over 65 percent of evaluated tasks.

AI Coding Agent Security Benchmark news update: who is affected

Software developers and development organizations utilizing commercial autonomous harnesses are directly impacted by these results. For example, teams using Cursor with Claude Opus 4.7, evaluated on April 17, 2026, receive a 79.9 percent functional completion rate alongside an 18.4 percent security correctness rate. Similarly, engineers running Claude Code with Claude Sonnet 5, benchmarked on July 2, 2026, experience an 83.2 percent functional success rate accompanied by a 19.6 percent security rate. Older configurations like Cursor with Gemini 3 Pro from February 24, 2026, produced only a 5.6 percent security score.

Application security engineers and repository maintainers must reconsider automated merge processes that rely solely on passing unit tests. Because coding agents lack native security context, Endor Labs is introducing AURI, a security harness for coding agents that remains free for developers to enforce security checks from registries to workstations. Teams integrating Codex or Cursor directly into automated pull request workflows risk merging insecure implementations across multiple repository modules. Security teams must implement explicit security verification gates to catch Common Weakness Enumeration vulnerabilities before agent-generated code reaches production environments.

AI Coding Agent Security Benchmark news update: what to watch

The Agent Security League tracks commercial releases as new foundation models and developer harnesses enter the market. A key technical threshold to watch is whether upcoming models can cross the 40 percent security correctness boundary, which no model achieved across the first 33 evaluations. Observers should track the competitive dynamic between Claude Code and Codex, which hold the top five positions on the board. While Codex achieved 34.6 percent security with GPT-6 Astra and 34.1 percent with GPT-6.1 Sol, Claude Code leads with Claude Fable 5.1 at 37.4 percent and Claude Opus 5.5 at 33.5 percent.

Future updates will clarify how effectively prompt hardening and cheating detection counter emerging model evasion strategies. As autonomous agents take on broader development responsibilities beyond the 200 Python tasks and 77 Common Weakness Enumeration classes from SusVibes, verification frameworks must expand across multi-language enterprise repositories. Engineering teams should monitor adoption of developer security harnesses like AURI to inspect generated code prior to merging. Until AI coding agents incorporate comprehensive security context during synthesis, organizations must treat functional execution and security correctness as completely distinct operational standards.