Home Posts API Design Reference: Idempotency & Outbox Patterns [2026]
Developer Reference

API Design Reference: Idempotency & Outbox Patterns [2026]

API Design Reference: Idempotency & Outbox Patterns [2026]
Dillip Chowdary
Dillip Chowdary
Tech Entrepreneur & Innovator · May 03, 2026 · 11 min read

Bottom Line

Idempotency protects the request edge; transactional outbox protects the commit-to-event boundary. You usually need both if an API write can be retried and also emits downstream events.

Key Takeaways

  • Safe retries need a request fingerprint and stored response, not just a UUID header.
  • Write domain data and outbox rows in one transaction; publish only after commit.
  • Consumers still must be idempotent because relays and brokers are usually at-least-once.
  • Use FOR UPDATE SKIP LOCKED and partial indexes to scale pollers cleanly.
  • Key retention must outlive your retry window; public APIs commonly keep keys for 24h to 7d.

Per RFC 9110, PUT, DELETE, and safe methods are idempotent by semantics, but most business writes still arrive as POST. That leaves two common failure windows: duplicate request execution at the API edge and dual-write inconsistency between your database and event bus. This reference compresses the patterns, tables, commands, and guardrails you need to implement both without pretending you can buy “exactly once” from a broker checkbox.

Quick Reference

Bottom Line

Use idempotency to replay the same API result for the same logical write, and use transactional outbox to atomically persist the write and the event intent. If either side is missing, retries or publish failures will eventually create duplicates or gaps.

ConcernIdempotency PatternTransactional OutboxEdge
Duplicate client retriesStores a key, request fingerprint, and replayable responseDoes not solve request replayIdempotency
DB write plus event publishDoes not solve dual writesCommits domain row and event row togetherOutbox
Concurrent duplicatesNeeds uniqueness and in-flight handlingNeeds consumer dedupe anywayTie
Ordering guaranteesUsually irrelevant at request edgeMust preserve aggregate order explicitlyOutbox
Failure replay behaviorReturns first stored result for the same keyRelays committed events laterTie

What to persist

  • Idempotency store: tenant scope, operation scope, key, canonical request hash, response status, response body, expiry.
  • Outbox row: aggregate type, aggregate ID, event type, payload, sequence, created time, published time.
  • Consumer dedupe: message ID or domain-level uniqueness guard.

When to choose which

Choose idempotency when:

  • Your clients retry on timeout, disconnect, or ambiguous network failure.
  • The operation is exposed as POST or PATCH.
  • You need the same logical request to return the same logical result.

Choose transactional outbox when:

  • A committed write must trigger a message, webhook, or event.
  • You cannot rely on two-phase commit across the database and broker.
  • You need durable publish intent even when the process dies after commit.
Watch out: An idempotency key is not a business key. Reusing the same key with a different payload should fail fast, not silently create a second semantic operation.

Live Search Filter

For Type B references, a live filter keeps the page usable once the command list grows. The pattern below filters any row marked with data-filter-item and focuses the search box with /.

ItemTags
Replay-safe POSTidempotency retry response cache
Outbox pollerrelay queue publisher batch
Consumer dedupeprocessed messages conflict insert
SKIP LOCKEDpostgres concurrency poller
const input = document.getElementById('ref-filter-input');
const rows = [...document.querySelectorAll('[data-filter-item]')];

function applyFilter(value) {
  const q = value.trim().toLowerCase();
  rows.forEach((row) => {
    row.hidden = q !== '' && !row.textContent.toLowerCase().includes(q);
  });
}

input?.addEventListener('input', (e) => applyFilter(e.target.value));

document.addEventListener('keydown', (e) => {
  if (e.key === '/' && document.activeElement !== input) {
    e.preventDefault();
    input?.focus();
  }
  if (e.key === 'Escape' && document.activeElement === input) {
    input.value = '';
    applyFilter('');
    input.blur();
  }
});

Keyboard shortcuts

ShortcutActionWhy it matters
/Focus filterFast scan on long reference pages.
EscClear filterReturns the full command list immediately.
g cJump to commandsUseful during incident response.
g fJump to configurationUseful when changing schema or retention.
g aJump to advanced usageUseful when tuning pollers and consumers.

Commands by Purpose

These are reference commands, not a framework prescription. If you log request snapshots or event payloads during debugging, run them through the Data Masking Tool before sharing traces outside production access boundaries.

Generate or reuse an idempotency key

uuidgen
curl -X POST https://api.example.com/orders \
  -H 'Content-Type: application/json' \
  -H 'Idempotency-Key: 8e03978e-40d5-43e8-bc93-6894a57f9324' \
  -d '{
    "customerId": "cus_123",
    "amount": 4200,
    "currency": "USD"
  }'

Verify replay behavior

curl -X POST https://api.example.com/orders \
  -H 'Content-Type: application/json' \
  -H 'Idempotency-Key: 8e03978e-40d5-43e8-bc93-6894a57f9324' \
  -d '{
    "customerId": "cus_123",
    "amount": 4200,
    "currency": "USD"
  }'
curl -X POST https://api.example.com/orders \
  -H 'Content-Type: application/json' \
  -H 'Idempotency-Key: 8e03978e-40d5-43e8-bc93-6894a57f9324' \
  -d '{
    "customerId": "cus_123",
    "amount": 4200,
    "currency": "USD"
  }'
  • The second call should replay the first logical result.
  • If the payload changes under the same key, return a conflict-style error.

Inspect an outbox backlog

SELECT id, aggregate_type, aggregate_id, event_type, created_at
FROM outbox
WHERE published_at IS NULL
ORDER BY created_at
LIMIT 50;

Claim a relay batch safely

SELECT id, aggregate_id, event_type, payload
FROM outbox
WHERE published_at IS NULL
ORDER BY created_at
FOR UPDATE SKIP LOCKED
LIMIT 100;

Mark rows as published

UPDATE outbox
SET published_at = now()
WHERE id = ANY($1);

Consumer-side dedupe

INSERT INTO processed_messages (consumer_name, message_id, processed_at)
VALUES ($1, $2, now())
ON CONFLICT DO NOTHING;
  • If the insert affects zero rows, the message was already handled.
  • Do domain writes only after the dedupe insert succeeds.

Configuration

Idempotency table

CREATE TABLE api_idempotency (
  tenant_id text NOT NULL,
  operation text NOT NULL,
  idem_key text NOT NULL,
  request_hash text NOT NULL,
  response_status integer,
  response_body jsonb,
  resource_id text,
  created_at timestamptz NOT NULL DEFAULT now(),
  expires_at timestamptz NOT NULL,
  PRIMARY KEY (tenant_id, operation, idem_key)
);

CREATE INDEX api_idempotency_expires_idx
  ON api_idempotency (expires_at);

Outbox table

CREATE TABLE outbox (
  id bigint GENERATED ALWAYS AS IDENTITY PRIMARY KEY,
  aggregate_type text NOT NULL,
  aggregate_id text NOT NULL,
  event_type text NOT NULL,
  sequence bigint NOT NULL,
  payload jsonb NOT NULL,
  headers jsonb NOT NULL DEFAULT '{}'::jsonb,
  created_at timestamptz NOT NULL DEFAULT now(),
  available_at timestamptz NOT NULL DEFAULT now(),
  published_at timestamptz
);

CREATE INDEX outbox_unpublished_idx
  ON outbox (available_at, id)
  WHERE published_at IS NULL;

Application defaults

  • Key scope: include tenant and operation, not just the raw header value.
  • Fingerprint: hash a canonical request body plus route-level invariants.
  • Expiry: keep keys longer than the maximum client retry horizon; public API practice commonly ranges from at least 24 hours to 7 days.
  • Stored result: persist status and body only after execution begins; validation failures usually should not occupy the key permanently.
  • Outbox ordering: sequence per aggregate if consumers depend on strict business order.

Minimal write transaction

BEGIN;

INSERT INTO orders (id, customer_id, amount, currency, status)
VALUES ($1, $2, $3, $4, 'accepted');

INSERT INTO outbox (aggregate_type, aggregate_id, event_type, sequence, payload)
VALUES ('order', $1, 'order.accepted', $5, $6::jsonb);

COMMIT;
Pro tip: Keep the idempotency store and business write in the same primary database when possible. It simplifies consistency and removes one more distributed failure mode from the hot path.

Advanced Usage

Concurrent duplicates and in-flight requests

  • First request creates or locks the idempotency record and becomes the owner of execution.
  • Second request with the same key should either wait briefly, return an in-progress response, or replay once the first result is durable.
  • Never let two workers independently execute the same logical operation because they saw the same key at nearly the same time.

Canonical request hashing

  • Normalize JSON field ordering before hashing.
  • Exclude transport-only headers that should not change operation meaning.
  • Include tenant, route, and business invariants so the same key cannot cross logical boundaries.

Polling relay vs CDC

  • Polling relay is simpler to understand and works well with relational tables.
  • CDC removes table scans but ties you more tightly to database-specific streaming infrastructure.
  • AWS guidance explicitly treats both as valid outbox implementations; choose based on operating model, not ideology.

Consumer idempotency is still mandatory

  • Even if your relay marks rows after publish, a crash between publish and mark can emit duplicates.
  • Even if the broker claims strong delivery semantics, downstream handlers still face retries and redeliveries.
  • The practical target is usually effectively once at the business boundary, not metaphysical exactly-once transport.

Reference pseudo-flow

1. Receive POST with tenant scope and Idempotency-Key.
2. Canonicalize request and compute request_hash.
3. Insert idempotency row or load existing row by (tenant, operation, key).
4. If existing hash differs, reject.
5. If existing completed result exists, replay it.
6. If no result exists, execute domain write and insert outbox row in one transaction.
7. Persist the response on the idempotency row.
8. Relay committed outbox rows asynchronously.
9. Consumer inserts message_id into processed_messages with ON CONFLICT DO NOTHING.
10. Only the first successful consumer insert performs domain side effects.

Frequently Asked Questions

How do I make a POST endpoint idempotent without changing it to PUT? +
Keep the POST semantics and add an idempotency layer keyed by tenant, operation, and a client-supplied Idempotency-Key. Store a canonical request hash plus the first durable response, and replay that response when the same key arrives again with the same payload.
Should I store failed responses in the idempotency table? +
Store results only after endpoint execution actually begins. That usually means validation failures and some concurrency conflicts are retriable, while a completed application failure can be replayed if you want the same logical result returned for the same key.
Does transactional outbox guarantee exactly-once delivery? +
No. It guarantees atomic persistence of the business write and the publish intent, which removes the classic dual-write gap. You still need idempotent consumers because relays and brokers commonly deliver at least once.
When should I use CDC instead of polling the outbox table? +
Use CDC when your platform already operates database change streams well and you want lower polling overhead. Use table polling when you want a simpler, more portable design that is easy to debug with plain SQL.

Get Engineering Deep-Dives in Your Inbox

Weekly breakdowns of architecture, security, and developer tooling — no fluff.

Found this useful? Share it.