Stateful MBT can uncover invalid tool-call sequences before prod. Learn to model, generate, and verify AI agent workflows in Python. Read now.
Why AI agent workflows need a different kind of testing
An AI agent is not a pure function. It carries state across turns, decides which tool to call next based on prior results, and can reach the same goal through many different sequences of actions. Example-based unit tests only check the paths you thought to write down, which means the dangerous cases — a tool called before its prerequisite, an action repeated when it should run once, a sequence that leaves the system in a contradictory state — usually slip through until production traffic finds them.
Model-based testing (MBT) attacks this from the other direction. Instead of enumerating individual cases, you describe the rules the agent must obey as a model, then let a generator produce sequences of tool calls that explore that model. Because the sequences are generated rather than handwritten, they surface orderings you would never have thought to try.
Modeling the agent as a state machine
The core of stateful MBT is a model of the agent's world as states and transitions. Each tool the agent can call is a transition with a precondition (when is this call legal?) and an effect (how does calling it change the state?). The model does not reproduce the agent's logic; it captures the invariants the agent must never violate.
Concretely, for each tool you specify:
- Preconditions — what must already be true in the state for this call to be valid (for example, a session must be opened before a query runs).
- State update — how a successful call moves the model forward.
- Postconditions and invariants — properties that must hold after the call, and across the whole run regardless of path.
Generating and verifying sequences in Python
In Python, a property-based testing library is the natural engine for this. You define the tools as rules on a state-machine class, annotate each with its precondition, and let the library drive: it repeatedly picks a legal transition, applies it to both the model and the real agent, and checks that the agent's behavior stays consistent with the model's expectations. When a property breaks, the library shrinks the failing run to the shortest sequence that still reproduces the bug, which turns a sprawling failure into a minimal, readable repro.
The valuable part is that generation only ever proposes calls whose preconditions hold in the current model state, so every sequence is a plausible workflow rather than random noise. If the real agent tries to fire a tool the model considers illegal in that state, or ends a run with an invariant broken, the test fails — and you have caught an invalid tool-call sequence before a user did.
Making it practical
Start narrow. Model one workflow and its handful of tools, encode the two or three invariants that would cause real damage if violated, and grow the model as you learn which failure modes matter. Keep the model readable, because it doubles as executable documentation of how the agent is supposed to behave.
Run the generated suite in CI so regressions in prompt changes, tool definitions, or orchestration logic show up as failing sequences rather than incidents. When a shrunk counterexample appears, add it as a fixed regression test alongside the generative suite, so the specific bug stays pinned even as the model keeps exploring new territory.