Idempotency Is a Feature, Not a Nice-to-Have
Why retry-safe endpoints need to be designed in from the start, not bolted on after the first duplicate payment
The first time a payment went through twice in a system I was building, it was my fault. Not because I wrote bad code — the code was fine in isolation. It was my fault because I treated idempotency as something to add later, after the "real" functionality was working.
There is no later. Idempotency is a first-class design concern, and treating it as a polish step is how you end up debugging duplicate charges at 11pm.
What Idempotency Actually Means
An operation is idempotent if running it more than once produces the same result as running it once. In the context of API endpoints and background jobs, that means: if the same request arrives twice — whether because of a network retry, a webhook replay, or a user double-clicking a button — the system behaves as if it arrived once.
This sounds simple. It is not simple.
The tricky part is that most of the work in a typical endpoint is side-effecting: charge a card, send an email, create a database record, trigger a third-party sync. Those are the exact operations that go wrong when the same request fires twice.
Webhooks Are the Common Case
Stripe's documentation says it plainly: your webhook endpoint will receive the same event more than once. This is not an edge case or a bug. It is a documented behavior, and every payment provider behaves similarly. Networks are unreliable, retries are automatic, and Stripe has no way of knowing whether your 500 response meant "I processed this and then crashed" or "I never saw this."
The naive approach is to process the event and return 200. That works right up until the moment it does not.
The robust approach has two parts. First, log the event ID as the first action — before you do anything else. Second, check for that ID at the top of your handler. If it exists, return 200 immediately. Do nothing. The event already happened.
// Pseudo-code, but the shape matters const event = await stripe.webhooks.constructEvent(body, sig, secret);
const existing = await db.webhookEvents.findUnique({ where: { eventId: event.id } }); if (existing) return { status: 200 }; // Already processed
await db.webhookEvents.create({ data: { eventId: event.id, type: event.type } });
// Now do the actual work await handleEvent(event);
The database write being first is not an accident. If your handler succeeds but the response never reaches Stripe, Stripe retries. If you write last, you risk processing the event twice before writing the guard. Write first, process second.
Payments Are the High-Stakes Version
Webhook deduplication handles retries from the provider's side. But what about retries from the user's side?
A user clicks "Pay." The request goes out. The network times out. The user clicks again. On your server, the first request may still be in-flight — or may have completed. You have no way to know which, and neither does the user.
Idempotency keys solve this. The client generates a unique key for the intent (not the request), includes it in the header, and the server uses it to detect and short-circuit duplicates. Stripe uses exactly this pattern for their payment intents API: you pass Idempotency-Key: <uuid>, and Stripe caches the response for 24 hours. If the same key arrives again, Stripe returns the cached response without re-processing.
Implementing this server-side requires caching at the right layer. The key should be tied to the operation — not the HTTP session, not the user ID alone — and the cached result should be stored before the operation completes (or at minimum atomically with it). The goal is that any client seeing a timeout can retry safely, forever, and eventually get the result.
For payment flows, I treat idempotency key generation as a client responsibility. The frontend generates a key at the moment the user initiates an action and holds onto it for the lifetime of that action. Retry attempts reuse the same key. The server returns the same result whether it is the first attempt or the tenth.
Subscription and Billing State Is Harder
One-off payments are conceptually cleaner than subscriptions. With subscriptions, state changes happen asynchronously and out of order. A customer.subscription.deleted event can arrive before customer.subscription.updated. A invoice.payment_succeeded can arrive multiple times for the same billing cycle if Stripe retries a failed webhook delivery.
The pattern that holds up here is treating webhook handlers as state reconcilers rather than event processors. Instead of "handle this event sequentially," the approach is "bring the stored state into agreement with what this event describes, regardless of what state I'm currently in."
For subscription status, that means something like: when I receive a subscription event, look at the subscription's status field and the current_period_end, and write those values to my database — unconditionally, with a timestamp check to ignore events older than what I already have. The event ID deduplication guard handles exact duplicates; the timestamp logic handles out-of-order delivery.
It is more code than a simple sequential handler. But it survives retries, replays, and redeliveries without breaking.
Imports and Bulk Operations
The idempotency problem appears in a different form with bulk data operations. If a user uploads a file and the import job runs twice — because the job crashed and was retried, because the user re-uploaded the same file, because a queue delivered the message twice — you usually want exactly one set of records created.
The approach depends on what "same" means for the operation. If records have natural unique identifiers in the source data, those become upsert keys. If they do not, you build them: a hash of the record contents, a combination of fields that should be unique, a user-supplied external ID.
The critical thing is that upserts must be atomic. If your "check then insert" logic runs twice concurrently, both checks can pass before either insert completes. Most databases have mechanisms for this: INSERT ... ON CONFLICT DO NOTHING, INSERT ... ON CONFLICT DO UPDATE, Prisma's upsert — but you have to use them, and you have to define the conflict target correctly.
I have seen import pipelines that check for duplicates with a query, do work, then insert. That is not idempotent. That is a race condition waiting for production load to reveal it.
The Design Question
None of these patterns are hard to implement once you are looking for them. The problem is that they are invisible during development. Test environments do not retry webhooks. Users in demos do not double-click. Network timeouts do not happen in your localhost.
The failure mode surfaces in production, at scale, under real conditions. And when it does, you are debugging state that is already wrong rather than building a system that handles the case correctly.
The question I now ask upfront, for any endpoint that writes data: what happens if this runs twice? If the answer is "something bad," that is a design gap, not an acceptable risk to revisit later. Idempotency constraints shape the data model, the error-handling logic, and sometimes the API contract. They are not a layer you can add over a completed system.
Design for retry-safety from the start. Assume the network will lie to you. It will.