API Design Reference: Idempotency & Outbox Patterns [2026]
Bottom Line
Idempotency protects the request edge; transactional outbox protects the commit-to-event boundary. You usually need both if an API write can be retried and also emits downstream events.
Key Takeaways
- ›Safe retries need a request fingerprint and stored response, not just a UUID header.
- ›Write domain data and outbox rows in one transaction; publish only after commit.
- ›Consumers still must be idempotent because relays and brokers are usually at-least-once.
- ›Use FOR UPDATE SKIP LOCKED and partial indexes to scale pollers cleanly.
- ›Key retention must outlive your retry window; public APIs commonly keep keys for 24h to 7d.
Per RFC 9110, PUT, DELETE, and safe methods are idempotent by semantics, but most business writes still arrive as POST. That leaves two common failure windows: duplicate request execution at the API edge and dual-write inconsistency between your database and event bus. This reference compresses the patterns, tables, commands, and guardrails you need to implement both without pretending you can buy “exactly once” from a broker checkbox.
Quick Reference
Bottom Line
Use idempotency to replay the same API result for the same logical write, and use transactional outbox to atomically persist the write and the event intent. If either side is missing, retries or publish failures will eventually create duplicates or gaps.
| Concern | Idempotency Pattern | Transactional Outbox | Edge |
|---|---|---|---|
| Duplicate client retries | Stores a key, request fingerprint, and replayable response | Does not solve request replay | Idempotency |
| DB write plus event publish | Does not solve dual writes | Commits domain row and event row together | Outbox |
| Concurrent duplicates | Needs uniqueness and in-flight handling | Needs consumer dedupe anyway | Tie |
| Ordering guarantees | Usually irrelevant at request edge | Must preserve aggregate order explicitly | Outbox |
| Failure replay behavior | Returns first stored result for the same key | Relays committed events later | Tie |
What to persist
- Idempotency store: tenant scope, operation scope, key, canonical request hash, response status, response body, expiry.
- Outbox row: aggregate type, aggregate ID, event type, payload, sequence, created time, published time.
- Consumer dedupe: message ID or domain-level uniqueness guard.
When to choose which
Choose idempotency when:
- Your clients retry on timeout, disconnect, or ambiguous network failure.
- The operation is exposed as POST or PATCH.
- You need the same logical request to return the same logical result.
Choose transactional outbox when:
- A committed write must trigger a message, webhook, or event.
- You cannot rely on two-phase commit across the database and broker.
- You need durable publish intent even when the process dies after commit.
Live Search Filter
For Type B references, a live filter keeps the page usable once the command list grows. The pattern below filters any row marked with data-filter-item and focuses the search box with /.
| Item | Tags |
|---|---|
| Replay-safe POST | idempotency retry response cache |
| Outbox poller | relay queue publisher batch |
| Consumer dedupe | processed messages conflict insert |
| SKIP LOCKED | postgres concurrency poller |
const input = document.getElementById('ref-filter-input');
const rows = [...document.querySelectorAll('[data-filter-item]')];
function applyFilter(value) {
const q = value.trim().toLowerCase();
rows.forEach((row) => {
row.hidden = q !== '' && !row.textContent.toLowerCase().includes(q);
});
}
input?.addEventListener('input', (e) => applyFilter(e.target.value));
document.addEventListener('keydown', (e) => {
if (e.key === '/' && document.activeElement !== input) {
e.preventDefault();
input?.focus();
}
if (e.key === 'Escape' && document.activeElement === input) {
input.value = '';
applyFilter('');
input.blur();
}
});
Keyboard shortcuts
| Shortcut | Action | Why it matters |
|---|---|---|
/ | Focus filter | Fast scan on long reference pages. |
Esc | Clear filter | Returns the full command list immediately. |
g c | Jump to commands | Useful during incident response. |
g f | Jump to configuration | Useful when changing schema or retention. |
g a | Jump to advanced usage | Useful when tuning pollers and consumers. |
Commands by Purpose
These are reference commands, not a framework prescription. If you log request snapshots or event payloads during debugging, run them through the Data Masking Tool before sharing traces outside production access boundaries.
Generate or reuse an idempotency key
uuidgen
curl -X POST https://api.example.com/orders \
-H 'Content-Type: application/json' \
-H 'Idempotency-Key: 8e03978e-40d5-43e8-bc93-6894a57f9324' \
-d '{
"customerId": "cus_123",
"amount": 4200,
"currency": "USD"
}'
Verify replay behavior
curl -X POST https://api.example.com/orders \
-H 'Content-Type: application/json' \
-H 'Idempotency-Key: 8e03978e-40d5-43e8-bc93-6894a57f9324' \
-d '{
"customerId": "cus_123",
"amount": 4200,
"currency": "USD"
}'
curl -X POST https://api.example.com/orders \
-H 'Content-Type: application/json' \
-H 'Idempotency-Key: 8e03978e-40d5-43e8-bc93-6894a57f9324' \
-d '{
"customerId": "cus_123",
"amount": 4200,
"currency": "USD"
}'
- The second call should replay the first logical result.
- If the payload changes under the same key, return a conflict-style error.
Inspect an outbox backlog
SELECT id, aggregate_type, aggregate_id, event_type, created_at
FROM outbox
WHERE published_at IS NULL
ORDER BY created_at
LIMIT 50;
Claim a relay batch safely
SELECT id, aggregate_id, event_type, payload
FROM outbox
WHERE published_at IS NULL
ORDER BY created_at
FOR UPDATE SKIP LOCKED
LIMIT 100;
Mark rows as published
UPDATE outbox
SET published_at = now()
WHERE id = ANY($1);
Consumer-side dedupe
INSERT INTO processed_messages (consumer_name, message_id, processed_at)
VALUES ($1, $2, now())
ON CONFLICT DO NOTHING;
- If the insert affects zero rows, the message was already handled.
- Do domain writes only after the dedupe insert succeeds.
Configuration
Idempotency table
CREATE TABLE api_idempotency (
tenant_id text NOT NULL,
operation text NOT NULL,
idem_key text NOT NULL,
request_hash text NOT NULL,
response_status integer,
response_body jsonb,
resource_id text,
created_at timestamptz NOT NULL DEFAULT now(),
expires_at timestamptz NOT NULL,
PRIMARY KEY (tenant_id, operation, idem_key)
);
CREATE INDEX api_idempotency_expires_idx
ON api_idempotency (expires_at);
Outbox table
CREATE TABLE outbox (
id bigint GENERATED ALWAYS AS IDENTITY PRIMARY KEY,
aggregate_type text NOT NULL,
aggregate_id text NOT NULL,
event_type text NOT NULL,
sequence bigint NOT NULL,
payload jsonb NOT NULL,
headers jsonb NOT NULL DEFAULT '{}'::jsonb,
created_at timestamptz NOT NULL DEFAULT now(),
available_at timestamptz NOT NULL DEFAULT now(),
published_at timestamptz
);
CREATE INDEX outbox_unpublished_idx
ON outbox (available_at, id)
WHERE published_at IS NULL;
Application defaults
- Key scope: include tenant and operation, not just the raw header value.
- Fingerprint: hash a canonical request body plus route-level invariants.
- Expiry: keep keys longer than the maximum client retry horizon; public API practice commonly ranges from at least 24 hours to 7 days.
- Stored result: persist status and body only after execution begins; validation failures usually should not occupy the key permanently.
- Outbox ordering: sequence per aggregate if consumers depend on strict business order.
Minimal write transaction
BEGIN;
INSERT INTO orders (id, customer_id, amount, currency, status)
VALUES ($1, $2, $3, $4, 'accepted');
INSERT INTO outbox (aggregate_type, aggregate_id, event_type, sequence, payload)
VALUES ('order', $1, 'order.accepted', $5, $6::jsonb);
COMMIT;
Advanced Usage
Concurrent duplicates and in-flight requests
- First request creates or locks the idempotency record and becomes the owner of execution.
- Second request with the same key should either wait briefly, return an in-progress response, or replay once the first result is durable.
- Never let two workers independently execute the same logical operation because they saw the same key at nearly the same time.
Canonical request hashing
- Normalize JSON field ordering before hashing.
- Exclude transport-only headers that should not change operation meaning.
- Include tenant, route, and business invariants so the same key cannot cross logical boundaries.
Polling relay vs CDC
- Polling relay is simpler to understand and works well with relational tables.
- CDC removes table scans but ties you more tightly to database-specific streaming infrastructure.
- AWS guidance explicitly treats both as valid outbox implementations; choose based on operating model, not ideology.
Consumer idempotency is still mandatory
- Even if your relay marks rows after publish, a crash between publish and mark can emit duplicates.
- Even if the broker claims strong delivery semantics, downstream handlers still face retries and redeliveries.
- The practical target is usually effectively once at the business boundary, not metaphysical exactly-once transport.
Reference pseudo-flow
1. Receive POST with tenant scope and Idempotency-Key.
2. Canonicalize request and compute request_hash.
3. Insert idempotency row or load existing row by (tenant, operation, key).
4. If existing hash differs, reject.
5. If existing completed result exists, replay it.
6. If no result exists, execute domain write and insert outbox row in one transaction.
7. Persist the response on the idempotency row.
8. Relay committed outbox rows asynchronously.
9. Consumer inserts message_id into processed_messages with ON CONFLICT DO NOTHING.
10. Only the first successful consumer insert performs domain side effects.
Frequently Asked Questions
How do I make a POST endpoint idempotent without changing it to PUT? +
POST semantics and add an idempotency layer keyed by tenant, operation, and a client-supplied Idempotency-Key. Store a canonical request hash plus the first durable response, and replay that response when the same key arrives again with the same payload.Should I store failed responses in the idempotency table? +
Does transactional outbox guarantee exactly-once delivery? +
When should I use CDC instead of polling the outbox table? +
Get Engineering Deep-Dives in Your Inbox
Weekly breakdowns of architecture, security, and developer tooling — no fluff.