Webhook Idempotency and Deduplication (2026): Stop Processing Events Twice

Why do webhooks arrive more than once?

Webhooks arrive more than once because providers retry any delivery they cannot confirm, and slow responses, lost acknowledgments, provider redeliveries and manual resends all look unconfirmed. Webhook providers face a choice when a delivery is ambiguous: deliver at-most-once (and silently drop events when things go wrong) or at-least-once (and occasionally send the same event twice). Every serious provider picks at-least-once, because a dropped payment.succeeded is worse than a duplicated one. That decision pushes the deduplication problem onto you, the receiver.

Retry schedules amplify this. If your endpoint flaps during an incident, a single event can arrive five or six times over hours. For the full picture of how providers back off and when, see our guide on webhook retries best practices.

Which ID do Stripe, GitHub and Shopify send for deduplication?

The good news: most providers give you a stable identifier for exactly this purpose. The details differ enough that it's worth being precise.

If your provider sends nothing usable — some internal services and smaller SaaS tools don't — you'll need to derive a key yourself. That's next.

What should you use as a webhook idempotency key?

Use the provider's event or delivery ID, prefixed with the source, and fall back to a SHA-256 of the raw body only when no ID exists. An idempotency key is the value you use to answer one question: “have I seen this exact event before?” Getting the key wrong breaks dedup in both directions — too broad and you drop distinct events, too narrow and duplicates slip through.

import { createHash } from "crypto";

// Fallback only: derive a key from the exact raw bytes.
// Re-serializing parsed JSON changes key order and whitespace,
// so two identical events can hash differently.
const idempotencyKey = createHash("sha256")
  .update(rawBody)
  .digest("hex");

How do you deduplicate webhooks in Postgres and Redis?

Claim the key with a single atomic write — a primary-key insert with ON CONFLICT DO NOTHING in Postgres or SET ... NX EX in Redis — and only process the event when the write succeeded. Two patterns cover the vast majority of real systems. Both hinge on the same property: the check and the claim must be a single atomic operation.

Pattern 1: Postgres unique constraint

Create a table whose primary key is the idempotency key, and let the database enforce uniqueness:

CREATE TABLE webhook_events (
  event_id    TEXT PRIMARY KEY,   -- "stripe:evt_..." scoped key
  source      TEXT NOT NULL,
  received_at TIMESTAMPTZ NOT NULL DEFAULT now()
);

In the handler, insert and let the conflict tell you whether this is a duplicate:

const inserted = await db.query(
  `INSERT INTO webhook_events (event_id, source)
   VALUES ($1, 'stripe')
   ON CONFLICT (event_id) DO NOTHING
   RETURNING event_id`,
  [`stripe:${event.id}`]
);

if (inserted.rowCount === 0) {
  // Duplicate. Acknowledge with a 200 so the provider stops retrying.
  return res.status(200).send("ok");
}

await processEvent(event);
return res.status(200).send("ok");

Postgres has no native TTL, so prune the table on a schedule — a nightly cron is fine:

DELETE FROM webhook_events
WHERE received_at < now() - INTERVAL '30 days';

Pattern 2: Redis SET NX with TTL

If you're already running Redis, SET ... NX EX gives you an atomic claim with automatic expiry in one command:

const key = `wh:stripe:${event.id}`;

// Atomic "set if not exists" with a 7-day TTL.
// Returns null if the key already existed.
const isFirstDelivery = await redis.set(key, "1", {
  NX: true,
  EX: 60 * 60 * 24 * 7,
});

if (!isFirstDelivery) {
  return res.status(200).send("ok"); // duplicate
}

await processEvent(event);
return res.status(200).send("ok");

Transactional dedup in Node.js and Python

The claim and the database side effects commit or roll back together. A concurrent duplicate's INSERT waits on the first transaction's row lock: if the first commits, the duplicate inserts nothing and skips; if it rolls back, the duplicate claims the key and does the work.

const client = await pool.connect();
try {
  await client.query("BEGIN");
  const { rowCount } = await client.query(
    `INSERT INTO webhook_events (event_id, source)
     VALUES ($1, 'stripe')
     ON CONFLICT (event_id) DO NOTHING`,
    [`stripe:${event.id}`]
  );
  if (rowCount === 1) {
    await applySideEffects(client, event); // same transaction
  }
  await client.query("COMMIT");
  res.status(200).send("ok"); // first delivery or duplicate
} catch (err) {
  await client.query("ROLLBACK"); // releases the key
  res.status(500).send("retry"); // provider will redeliver
} finally {
  client.release();
}

def handle_event(conn, event: dict) -> None:
    with conn.transaction():  # commit on success, rollback on exception
        cur = conn.execute(
            """INSERT INTO webhook_events (event_id, source)
               VALUES (%s, 'stripe')
               ON CONFLICT (event_id) DO NOTHING""",
            (f"stripe:{event['id']}",),
        )
        if cur.rowcount == 0:
            return  # duplicate: already processed
        apply_side_effects(conn, event)  # same transaction

Emails and third-party API calls cannot roll back. Write them to an outbox table inside the same transaction and send them from a worker, passing your event ID as the downstream idempotency key.

Is deduplication enough, or do side effects need to be idempotent too?

You want both. Everything above is “dedup at the door”: reject duplicates before processing starts. The complementary strategy is making the processing itself idempotent, so that running it twice produces the same end state. The strongest systems do both — the door check handles the common case cheaply, and idempotent side effects catch whatever slips through (an expired TTL, a lost Redis key, a replay from six months later).

A pattern you'll see in a lot of codebases — and should not copy — is check-then-act:

// DON'T do this: race condition between the check and the insert
const existing = await db.query(
  "SELECT 1 FROM webhook_events WHERE event_id = $1",
  [event.id]
);

if (existing.rowCount > 0) {
  return res.status(200).send("ok");
}

// Two concurrent deliveries of the same event can BOTH reach
// this line, because neither saw the other's uncommitted insert.
await db.query("INSERT INTO webhook_events (event_id) VALUES ($1)", [
  event.id,
]);
await processEvent(event); // ...and both process the event.

This isn't theoretical. Providers retry on a timer, your handler runs on multiple instances behind a load balancer, and a retry can land while the original delivery is still in-flight. Two requests run the SELECT at the same moment, both see zero rows, both proceed. INSERT ... ON CONFLICT DO NOTHING wins because the database serializes the conflict: exactly one insert succeeds, and the loser finds out atomically via rowCount. The same logic applies to Redis: GET then SET is a race, SET NX is not.

Idempotent side effects extend the same thinking into your domain logic:

How do you test webhook duplicate handling? (Test it with Hooklistener)

Deliver one real event to your handler several times and check that it was processed once. Dedup logic is easy to write and easy to get subtly wrong, and the failure only shows up when a duplicate actually arrives. With Hooklistener the whole test takes four steps, from the dashboard or from your AI assistant through the Hooklistener MCP server:

1. Capture one real event. Point Stripe, GitHub or Shopify at a Hooklistener endpoint and trigger an event (for Stripe, stripe trigger payment_intent.succeeded).
2. Replay it repeatedly. Run hooklistener listen --endpoint <ENDPOINT_ID> --port 3000 and call replay_request with target: "cli" five times (each call takes its own idempotency key). Add signing_provider and a stored signing_secret_id to re-sign each replay so signature checks pass.
3. Save the regression cases. Use save_request_case with an assertion on status 200, then save a second case whose body has a fresh event ID.
4. Rerun on every change. Call run_endpoint_cases against localhost or staging and read the pass/fail report with wait_for_case_run: the duplicate must return 200 without new side effects, the new ID must be processed.

The sections below go deeper on each step.

Capture once, replay repeatedly

Point your provider at a Hooklistener endpoint (or run a CLI tunnel in front of your local server) and trigger one real event. Once it's captured, replay it against your handler as many times as you like — same headers, same body, same signature. The sequence to verify:

• • Replay 1: handler processes the event and responds 200
• • Replays 2–5: handler short-circuits as a duplicate, still responds 200, and responds noticeably faster (no processing happened)
• • Side effects: exactly one row, one email, one charge — check your database, not just the status codes

Replay also lets you modify the body and headers before sending. Change the event ID to a fresh value and replay again: your handler should treat it as a brand-new event. That two-sided test — same key dedupes, new key processes — catches the embarrassing failure mode where your dedup accidentally swallows everything.

Read the responses, not just the requests

Hooklistener records the HTTP response for each captured webhook — status code, headers, body (up to 10 KB), and whether it came from the endpoint's default response or a matched mock rule — in a Response tab on each request. The request list has a color-coded status column (green 2xx, blue 3xx, amber 4xx, red 5xx), so a dedup test reads at a glance: five identical requests, five green responses. If replay three is amber or red, your duplicate path is throwing instead of acknowledging — which means the provider would keep retrying forever. On paid plans you can also select two to five captured requests and run an AI comparison that breaks down shared patterns, differences, timing, and anomalies across them — useful when you're staring at near-identical retries trying to spot what changed.

Generate real provider retries

Replays test your logic; real retries test the provider's actual behavior — headers, timing, whether the payload is byte-identical across attempts. Configure a Hooklistener endpoint to return a custom 500 response, point Stripe or Shopify at it, and trigger an event. The provider sees the failure and begins its retry schedule, and every retry lands in your webhook inbox where you can confirm the event ID really is stable across attempts before you bet your dedup strategy on it.

Group duplicates by event ID with request threads

Once real retries are flowing, you want to see them side by side. A thread rule on the endpoint correlates captured requests by a field you choose — e.g. body.id for Stripe — so every delivery of the same event lands in one thread. From there:

• • Count the attempts the provider actually made for one event, and how far apart they were
• • Diff two deliveries with the deterministic request diff to confirm the body is byte-identical and only delivery headers (timestamps, signatures, attempt counters) changed
• • Check the responses your endpoint returned for each attempt, so you can see which failure kept the retries coming

Lock the behavior in with a replay suite

A dedup check that works today can regress with the next refactor. Save the captured event as a replay case with an assertion on the response status, add it to a suite alongside a copy with a fresh event ID, and rerun the suite whenever the handler changes: the duplicate must still return 200, and the new ID must still be processed. The same cases are reachable through the MCP tools if you want your AI assistant to run them for you. For fanning one captured event out to several handlers at once, see sending webhooks to multiple endpoints. Plan details are on the pricing page.

To be clear about the boundary: Hooklistener captures, groups, and replays duplicates — it doesn't dedup them for you. Keep the dedup check in your own infrastructure where it can share a transaction with your side effects — and use Hooklistener to prove it works before you ship.

Idempotency Checklist

Frequently Asked Questions

Related Resources