What Is Agent Experience (Ax)? news update

Microsoft published an engineering guide outlining the emerging discipline of Agent Experience, establishing how software teams can systematically evaluate and optimize platforms for artificial intelligence tools. According to developer.microsoft.com's report, developers increasingly rely on automated assistants to consume application programming interfaces, software development kits, and command-line interfaces. When an automated system selects a deprecated endpoint or reports a faulty build as successful, the underlying product appears broken to human developers.

This analysis details how engineering organizations evaluate autonomous tool usage, resolve interface hurdles, and monitor total operational expenses across different artificial intelligence models. The reporting is designed for platform architects, documentation managers, and developer relations teams managing technical tooling. By auditing actual assistant behaviors across standardized test profiles, software publishers can prevent customer churn and identify whether specific product extensions introduce measurable operational drag.

01Propensity, efficacy, and task cost scores

Evaluating assistant interactions centers on measuring propensity and efficacy alongside overall task expenditure. Propensity tracks whether an assistant independently chooses a given technology during open-ended tasks that omit specific product names. Efficacy measures whether the model invokes current interfaces accurately, avoiding deprecated endpoints and outdated configurations once prompted with a named product.

Measurement frameworks frequently label propensity as discovery or selection while tracking efficacy as baseline quality. Teams assess every trial by weighing the final functional output against the total financial expense required to complete the task. Evaluating individual token rates without tracking full task completions frequently misleads engineering leadership regarding real production overheads.

Token price reductions do not guarantee lower runtime bills during multi-step developer operations. In SharePoint Framework project upgrade tests conducted across three scenarios and fifteen runs, Claude Sonnet 5 cost 3.7 times more per run than Sonnet 4.6 in GitHub Copilot Chat. That cost multiplication occurred despite Sonnet 5 offering a 33 percent lower per-token pricing schedule.

02Three layers of the AX stack

The practice of agent optimization requires understanding which architectural tiers software creators can influence. Engineering organizations cannot alter underlying model weights and can only influence application harnesses by negotiating with third-party vendors. Technical publishers maintain complete sovereignty over documentation repositories, custom agent extensions, and public programming interfaces.

Documentation updates deliver immediate operational results because active agents query source files while executing developer workflows. Unlike trained foundational models that require new pre-training iterations to assimilate recent facts, runtime agents parse documentation edits instantly. Maintaining synchronized technical notes ensures assistants discover operational modifications during their very next runtime pass.

Interface behaviors dictate whether programmatic tooling functions reliably during automated execution paths. Assistants interact with command-line tools, software development kits, and project scaffolders exactly like human engineers, trusting returned diagnostic messages. In one documented SharePoint Framework assessment, an unpinned project scaffolder created an environment rooted in July 2020 dependencies, yet the model registered the deployment as successful.

03Empirical tests of agent best practices

Platform teams must treat popular assistant optimizations as testable hypotheses rather than universally applicable rules. During automated SharePoint Framework upgrades in GitHub Copilot Chat with Claude Sonnet 4.6 on Windows, an unassisted model completed thirty of eighty configuration checks. Instructing the assistant to deploy the open-source CLI for Microsoft 365 improved success to seventy-five of eighty checks.

Relying on specialized assistant skills creates friction because human developers rarely discover or install auxiliary extensions. Microsoft addressed this gap by reviewing the agent trace, discovering that the assistant consistently inspected official SharePoint Framework release notes. Updating those release notes directly directed every subsequent execution to use the command-line utility without requiring developers to configure custom skills.

Several assumed optimizations degraded operational performance when verified under controlled environments. Forcing command-line interfaces into structured JSON input modes caused Claude Haiku 4.5 to fail three of five deployments while raising task costs up to elevenfold. Similarly, connecting the context7 Model Context Protocol documentation server yielded zero lift because tools failed to load in three of five runs.

04Core vocabulary and developer experience parallels

Systematic evaluation requires adopting consistent terminology to measure automated developer interactions. A bare baseline represents tasks run using only the foundational model and its base harness without supplemental extensions. Comparing experimental changes against this baseline reveals whether modifications introduce measurable task lift or create operational drag by degrading performance and inflating costs.

Every evaluation must define its specific agent profile, which catalogs the operating system, harness software, model versions, and active extensions. Results remain valid only for the precise profile tested, as differing runtime environments alter execution patterns. Tracking task costs alongside quality scores prevents organizations from adopting optimizations that improve success marginally while quadrupling token bills.

Agent optimization builds directly upon classical developer experience research and generative engine optimization. While search optimization focuses primarily on product discovery, agent testing examines whether assistants accurately complete technical workflows after selecting a product. Testing software with automated systems mirrors traditional usability observation, allowing teams to run identical multi-turn tasks repeatedly without burdening human engineering panels.

05Pragmatic five step evaluation cycle

Teams should initiate evaluation programs by gathering verbatim workflow requests from engineering teams. Test suites should include open-ended queries to measure discovery propensity as well as product-specific commands to measure functional efficacy. Running those initial tasks across an unadorned baseline reveals where automated systems encounter configuration roadblocks and documentation gaps.

Engineers should review runtime traces to identify failure points before writing specialized custom agent extensions. Correcting root causes directly within public documentation, application programming interfaces, and error outputs benefits every developer automatically without requiring optional add-ons. Teams must re-run standardized task suites following every interface adjustment to quantify changes in functional accuracy and token expenditures.

Organizations should scope all conclusions to the measured profile and repeat benchmarking whenever harness vendors release updated models. When companies already distribute Model Context Protocol servers or custom skills, teams must compare execution traces with and without those extensions. Adopting quantitative evaluations removes subjective guesswork, ensuring technical platforms maintain reliable interoperability with autonomous programming assistants.