The transactional outbox

The gap that loses money

Take the simplest possible version of an event-driven system: a service writes a row to its database, then publishes an event so other services can react — send a receipt, update a ledger, trigger a payout. That “then” is where things go wrong. The database commit and the message publish are two separate operations against two separate systems, and there is no way to make them atomic by wishing hard enough. If the process crashes, the container gets rescheduled, or the network to the message broker drops in the gap between them, you get a row in the database with no event ever published for it.

In a system with 170-plus event handlers driving charges, payouts, disputes and reconciliation off of these events, a silently dropped event isn’t a cosmetic bug. It’s a charge that never triggers a receipt, a dispute that never notifies the merchant, a payout that never gets recorded downstream. And because nothing threw an exception — the database write succeeded — nothing alerts you. You find out when a customer asks where their money went.

The pattern

The transactional outbox fixes this by refusing to treat the publish as a separate operation at all. Instead of writing the row and then publishing an event, the service writes the row and the event into the same database transaction, as two tables:

BEGIN;

INSERT INTO charges (id, amount, status, ...)
VALUES ($1, $2, 'succeeded', ...);

INSERT INTO outbox (id, event_type, payload, published_at)
VALUES ($3, 'charge.succeeded', $4, NULL);

COMMIT;

Both inserts succeed or both roll back — there’s no window where one happened and the other didn’t, because they’re the same transaction against the same database. A separate, dumb process — a relay — polls the outbox table for rows where published_at IS NULL, publishes them to the message broker, and marks them published:

async function relayOutboxEvents() {
  const pending = await db.query(
    `SELECT * FROM outbox WHERE published_at IS NULL ORDER BY id LIMIT 100 FOR UPDATE SKIP LOCKED`
  );
  for (const event of pending) {
    await broker.publish(event.event_type, event.payload);
    await db.query(`UPDATE outbox SET published_at = now() WHERE id = $1`, [event.id]);
  }
}

If the relay crashes after publishing but before marking the row published, the event gets published again on the next pass. That’s the trade-off the pattern makes deliberately: it guarantees at-least-once delivery, never zero-times delivery, in exchange for requiring every consumer to be idempotent.

The trade-off you’re accepting

At-least-once delivery means duplicates are a normal, expected occurrence, not an edge case. Every handler subscribing to these events has to treat a duplicate message as something that happens routinely and do nothing harmful when it does — usually by keying off the event’s id and recording which ids have already been processed before acting on one again:

async function handleChargeSucceeded(event: ChargeSucceededEvent) {
  const already = await db.query(
    `SELECT 1 FROM processed_events WHERE event_id = $1`, [event.id]
  );
  if (already.rowCount > 0) return;

  await db.transaction(async (tx) => {
    await recordLedgerEntry(tx, event);
    await tx.query(`INSERT INTO processed_events (event_id) VALUES ($1)`, [event.id]);
  });
}

This is more code than “just publish the event,” and it’s worth it. The alternative — trying to guarantee exactly-once delivery across a database and a message broker — runs into the same distributed-transaction problem the outbox exists to avoid, just moved one layer over.

Read/write split and the relay’s own load

At scale, the relay polling the outbox table competes for the same connections and locks as the write path that’s actively inserting new rows. Splitting reads from writes — the relay reading from a replica or a dedicated connection pool tuned for short polling queries, SKIP LOCKED to avoid contention between relay instances — keeps outbox delivery from becoming a bottleneck on the transactions producing the events in the first place. This also lets you run more than one relay instance for throughput without them fighting over the same rows.

What it bought during a bad afternoon

The value of this pattern shows up most clearly when something upstream is struggling — an API under load, a downstream consumer temporarily unavailable. Because events are durably recorded in the same transaction as the data producing them, a slow minute produces delayed side effects, never lost ones. Nothing needs manual reconciliation afterward because nothing was ever silently dropped; it was just queued a little longer than usual.

What this means for you

If your system publishes events after a database write and treats the publish as “probably fine,” you have a gap that will eventually lose one. The fix isn’t a more reliable message broker — it’s removing the gap entirely by writing the event in the same transaction as the data and letting a separate relay handle delivery. The cost is a table, a poller, and a discipline of idempotent handlers everywhere. The alternative cost is an event that quietly never happened.