Standard webhooks fail for long-running AI agents. Master the engineering of resilient async-API patterns using durable execution and SSE for 2026. Read now.
Why webhooks break for agentic work
Webhooks assume a short gap between request and result. You start a job, park a callback URL, and expect a single POST when the work finishes. Agentic tasks break that model. An agent may run for minutes or hours, pause for human approval, call tools in loops, fail midway, and resume later. A single terminal webhook cannot express progress, partial failure, or multi-step handoff. If the receiver is down when the callback fires, you lose the only signal the producer will send unless you build your own retry and reconciliation layer on both sides.
Fire-and-forget callbacks also hide operational truth. You cannot answer “where is this run?” from the callback alone. You need a durable job identity, an inspectable state machine, and a way for clients to subscribe to updates without inventing a one-off protocol per feature.
Model the task as durable execution, not a request
Treat each agent run as a long-lived workflow with a stable ID. Persist intent, inputs, step checkpoints, and outcomes before you acknowledge the start of work. The API that accepts a task should return immediately with that ID and a status resource—not block until the agent finishes. Subsequent steps write state transitions to storage you control: queued, running, waiting_for_input, succeeded, failed, cancelled. Retries then become replay of unfinished steps from the last durable checkpoint, not “hope the webhook arrives again.”
Durable execution also clarifies ownership. The producer owns step semantics and idempotency keys; the consumer owns how it reacts to state. Shared contracts matter: status enums, error shapes, and cancellation rules should be versioned like any other public API surface. Without that, every client reinvents polling heuristics and silently diverges.
Stream progress with SSE; reserve webhooks for terminal events
Server-Sent Events fit agent progress better than a single callback. The client opens a stream against the task ID and receives ordered events: step started, tool invoked, intermediate summary, waiting for approval, final result. SSE is unidirectional, reconnect-friendly with last-event IDs, and simple to proxy through standard HTTP infrastructure. For browser and dashboard UIs, that live feed is enough to show honest progress without short-lived WebSocket complexity.
Keep webhooks optional and narrow. Use them for terminal or high-value side effects—notify a third-party system when the run completes or when human input is required—backed by at-least-once delivery, signed payloads, and a dead-letter path. Clients that cannot hold an open stream should fall back to polling the status resource with exponential backoff, using ETags or updated-at fields so empty polls stay cheap.
- Accept task → return ID + status URL + optional stream URL
- Checkpoint every meaningful step before side effects that cannot be undone
- Expose SSE for progress; webhook only for completion or action-required
- Make cancel and resume first-class operations on the same task ID
Design for failure paths you will actually hit
Resilient async APIs assume disconnects, partial tool success, and operator intervention. Timeouts should move a run into a terminal or waiting state you can query, not leave it stuck as “running” forever. Idempotent step handlers prevent double charges when a worker retries after a crash. Correlate every log and metric with the task ID so on-call can reconstruct a single agent trajectory without grepping across services.
Ship the boring pieces first: a status schema, durable checkpoints, an SSE stream, and a documented recovery path. Once those hold under load and reconnects, optional webhooks become a convenience integration—not the backbone of agent reliability.