Avoiding Déjà Vu: Building Resilient APIs with Idempotency

Anyone who has spent a non-trivial amount of time on the internet will recognise the following scenario. You click a button to do something online, waited a second, nothing seems to happen, so you click it again... and now your social media post is duplicated, or you've got two orders in your cart. Welcome to the world of broken idempotency! Where your API gets a nasty sense of Déjà Vu, reliving the same moment over and over, while users keep paying the price. Déjà Vu, for anyone whose French is a bit rusty, means "seen before". For us humans, it can be a bit of a weird sensation in a conversation, but for our APIs, the consequences of reliving the same moment can be more extreme, both for us and our users.

In this post, we’re going to explore idempotency: what it is, why it matters, how to implement it safely, and what can go wrong when you don’t. Along the way, we’ll look at food delivery mishaps, mistaken identities, and the time Spotify’s login system accidentally handed the keys to someone else’s account.

This post is based on a talk I've given at the Dutch PHP and IPC Berlin conferences - if the scrollbar is looking a little bit small, and you'd rather watch the presentation, jump down here!

What Is Idempotency, Anyway?

Let’s start with the basics. Idempotency — pronounced eye-dem-po-ten-cy, though it’s often mangled in the wild! — is

a property of operations that can be repeated without changing the result beyond the first application

(thanks Wikipedia).

The idea is that if you apply an idempotent function once or fifty times, the result is the same. In practice, it means that making the same call over and over should be safe. Think of something like UPDATE users SET balance = 0, or strtolower(strtolower(strtolower('What A WeIRd IDeA'))). In both cases, the result is the same whether we're making the core call once, or 100 times.

A real world example is to imagine you're calling a lift. Whether you press the button once or ten times, only one lift is going to show up! That’s idempotency.

Why Idempotency Matters (Especially at Lunchtime)

Let’s say it’s nearly noon and you're ordering lunch through a food delivery app. You press “Order,” and your request is sent to the server. If everything goes well, the server processes the request, charges your card, and sends your lunch on its way.

That’s the happy path. But as developers know, most real-world requests don’t live exclusively on that path.

Maybe your device loses connection just as the server processes the request, and the response never makes it back. Or maybe your app retries the request automatically after a timeout. Or maybe, like most humans, you assume nothing happened and tap the button again. And again.

From the backend's perspective, that might look like multiple identical requests arriving in rapid succession. If your API isn’t idempotent, that could mean multiple charges, duplicate deliveries, and a very confusing lunchtime.

It’s Not Just Impatient Users

While double-clicking users are a frequent culprit, replayed requests can also arise from well-meaning retry logic. This is especially true in mobile apps, where connectivity is flaky and developers try to smooth over transient failures.

Beyond mobile, infrastructure can also be a cause of these replays. Load balancers might retry failed requests. CDNs may resend operations. Edge cases start to multiply, and if each one leads to duplicate transactions, refunds and chaos soon follow.

This is why every serious system — from AWS to Stripe to Google Cloud — treats idempotency as a core concern, especially for operations involving money or resource provisioning. One request should equal one action, no matter how many times it arrives.

Not All Verbs Are Created Equal

HTTP gives us a handy vocabulary for thinking about repeatable operations. Some methods, like GET, HEAD, and OPTIONS, are naturally idempotent — fetching the same resource twice won’t change its state.

Others, like POST and PATCH, are not. Sending a POST to create a new payment or a PATCH to increment a value can easily cause duplication if repeated. These are the operations where idempotency support matters most.

Even DELETE, which might seem idempotent (“you can only delete something once”), can behave oddly in UI-driven apps. If a DELETE request succeeds but the confirmation doesn’t arrive, retrying it might yield a confusing error like “User not found” — technically correct, but not exactly confidence-inspiring. This is why many idempotency implementations will treat verbs like DELETE and PUT as requiring idempotency support, when technically they're naturally idempotent. Their observable side-effects mean treating them similarly to POST and PUT is often a matter of improved UX and DX (developer experience).

Preventing Replays: A Tale of Two Lunches

Let’s walk through a typical failure scenario.

You place a food order. The request reaches the server, the charge goes through, and your lunch is dispatched — but the response gets lost. Maybe your device dropped off the network for a second. You wait. Nothing happens. So you try again.

The server sees a new request and processes it as usual. Another charge, another delivery.

Now you’re €24 lighter, and your colleague is wondering why you have two identical burritos.

What we want instead is for the server to say: “Hang on — I’ve seen this request before.” If it can recognize a repeat, it doesn’t need to reprocess it. It can just return the same response it did the first time, sparing your user from duplicate charges and you from refund requests.

That’s where idempotency keys come in.

Identifying Repeats (Without False Positives)

The hard part of building idempotency is figuring out: Have I seen this request before?

At first, you might think: why not just hash the request body? If two requests look the same, treat them as the same. But that quickly gets messy.

Imagine you're ordering a burger and chips to your office. Your teammate sees your screen and decides to order the same thing, to the same address. Their request body is identical to yours — but it's a different order, and shouldn't be deduplicated.

Now you're in collision territory.

Try adding the IP address? You’re still stuck if you're on shared Wi-Fi.

What about user ID? That works — until your marketing team says you need to support guest checkouts, with null user ids. Disambiguate on email addresses? The whole company is using accounts@example.com to simplify expense tracking of their lunches. Wherever we server-side engineers look, we're getting potential collisions in our attempts to uniquely identify requests.

The solution? Let the client help!

Let the Client Do the Hard Work

Instead of guessing whether a request is unique, we can make the client tell us. When it creates a request that might need retrying — say, a payment attempt — it should also generate a unique idempotency key and include it in the request headers.

If the same request is retried (by the client or infrastructure), it should include the same key. That way, the server can easily detect the replay.

In this example, the key is 1. In practice, this would be quite prone to collision! We'll shortly come on to key selection strategies.

Interestingly, most major API SDKs will support providing idempotency keys. Stripe, for example, lets you pass an Idempotency-Key header. What may be a surprise if you've used these APIs and never heard of idempotency, is that if you forget, their SDKs quietly generate a UUID behind the scenes. You might never know idempotency exists, but you’re being protected by it anyway.

And that’s part of the beauty here. With good SDK design, you can shield your users from the complexity while making your API more resilient.

// this is a little verbose, but makes v1 vs v2 behavior really clear
if (!$this->hasHeader($headers, 'Idempotency-Key')) {
  // all v2 requests should have an IK
  if ('v2' === $apiMode) {
    if ('post' === $method || 'delete' === $method) {
      $headers[] = 'Idempotency-Key: ' . $this->randomGenerator->uuid();
    }
  } else {
    // v1 requests should keep old behavior for consistency
    if ('post' === $method && Stripe::$maxNetworkRetries > 0) {
      $headers[] = 'Idempotency-Key: ' . $this->randomGenerator->uuid();
    }
  }
}

In the Stripe SDK case, we can see here the automatic application of idempotency keys to POST requests, as well as DELETE. As we discussed earlier, DELETE is technically idempotent naturally, but adding support here adds application-based idempotency support, similar to what is being provided for POST.

Implementing It: Middleware to the Rescue

So, how do we actually add idempotency support in a typical backend?

One of the cleanest ways is to use middleware — a wrapper around your request handler that can intercept requests, inspect headers, and decide what to do before your core application ever sees them.

If you’re using Laravel, Express, or pretty much any modern framework, middleware is your friend.

When a request comes in, your middleware can:

• Check if the method (POST, PATCH, etc) needs idempotency handling.
• Extract the idempotency key from the header.
• Look it up in a cache (like Redis) to see if a response already exists.
• If it does, return the cached response.
• If not, process the request as usual, then store the response using the idempotency key.

class IdempotencyMiddleware
{
  public function handle(Request $request, Closure $closure)
  {
    if (in_array($request->method(), $this->getIdempotentVerbs())) {
      return $next($request);
    }

    $idempotencyKey = $request->header('X-Idempotency-Key');
    $cacheKey = $this->buildCacheKey(auth()->user, $idempotencyKey);

    if (Cache::has($cacheKey)) {
      return $this->handleCachedResponse($cacheKey, $request->path());
    }

    return $this->processRequest($request, $cacheKey, $next);
  }
}

It’s simple, contained, and — importantly — your application logic doesn’t need to change. You don’t need to refactor your endpoints or sprinkle if-statements everywhere. The middleware handles it.

Protecting your clients, sometimes from themselves

A common question here is why we're passing the $request->path() around when returning a cached response. This is a way for us to protect our API-consuming clients from likely mistakes.

Within our cache-handling function, when we retrieve the value for a given idempotency key, we'll check what the original path the request was made on, and check it against the current path. They should match - should! But if our client is accidentally re-using idempotency keys in their implementation, detecting this and throwing an error can bring this mistake to light sooner rather than later.

For example:

• Client send key abcd1234, calls POST /users, creates user.
• The idempotency key abcd1234 now contains a response for a user creation.
• Client then calls POST /orders, but sends abcd1234 again.
• We retrieve the cached value for abcd1234, and notice that it's a response to /users, not /orders.
• We can throw an exception to alert our client to likely misconfiguration of their application.

How Long Should You Cache for?

Once you’ve decided to store responses keyed by an idempotency token, the next question becomes: for how long?

There’s no single right answer. Like all the most unsatisfying answers to simple questions, "it depends" — on the nature of your app, the likelihood of retries, and the business impact of duplication.

The options broadly break down as follows:

• ⏳   Short TTL (Mins - Hours)
• ⏱️   Medium TTL (Hours to Days)
• 📅   Long TTL (Days to Weeks)
• 🔒   Infinite TTL (Persistent Storage)

For fast-moving apps like food delivery or taxi hailing, retries are typically short-lived. A user might walk out of coverage, then regain signal and retry within a minute or two. In that case, a cache duration of a few minutes is often plenty.

Other cases need more breathing room. Imagine an invoicing system that runs overnight, then retries failed attempts the next day. Here, you might want to keep idempotency keys alive for 24 to 48 hours. This is typically the window Stripe's idempotency cache falls into.

And sometimes, the stakes are higher. A user might fail to upgrade their subscription due to a backend hiccup, reach out to support, and then retry the operation days later. In that scenario, you’re likely dealing with human-in-the-loop resolution, and a longer TTL can offer important protection.

Finally, in rare but highly regulated systems — financial services, government, healthcare — you might even opt for infinite TTLs. That usually means storing data in a persistent database rather than an ephemeral cache, and being very careful about your uniqueness logic.

As a rule of thumb: the higher the potential impact of duplication, the longer the TTL you’ll want.

But here’s the trick — the longer you keep entries, the higher the risk of a cache collision. And that brings us back to how we build our keys.

Building Better Cache Keys (Without Shooting Yourself in the Foot)

It’s easy to assume a UUID is good enough to prevent collisions, and for many systems, it is. But if your TTLs stretch into days or weeks, or your scale starts to spike, you may want to add extra disambiguation.

That might include:

• The user ID, if one exists.
• A timestamp or prefix to namespace the key by resource type.
• A checksum or signature based on the action being performed.

It’s a balancing act: you want keys to uniquely identify the intention behind a request, without being so brittle that a minor difference (like a reordered JSON field) causes a cache miss.

One small but helpful trick is to include the idempotency key itself in your response — perhaps as a custom header. This makes it easier to verify, during development or in logs, whether a response came from fresh processing or from the cache. Stripe and others use this approach, and it can save you a lot of head-scratching in staging environments.

Race Conditions

Caching alone isn’t enough. There’s another risk lurking: race conditions.

Imagine two identical requests arrive at your server a few milliseconds apart — one due to a retry, the other due to a network hiccup. Both check the cache. Neither sees an entry yet. Both go ahead and start processing.

Now you’ve got duplicate charges again, despite your best efforts.

To prevent this, you need a lock — a temporary ownership flag that says: “I’ve got this one.” The first request to arrive grabs the lock, processes the request, and then stores the result. The second request sees the lock, waits, and then — once the result is available — serves the cached response.

This is a well-known pattern in concurrent programming, and it’s easy to implement with tools like Redis. The key is to ensure that your lock automatically expires — in case the process dies unexpectedly — and that waiting requests eventually give up rather than waiting forever.

$lock = Cache::lock($cacheKey);

// Can't get a lock? Someone working on it!
// Wait it out
if (!$lock->get()) {
  return $this->waitForCacheLock($cacheKey, $request);
}

// We have the lock, so do the processing
$response = $this->processRequest($request, $cacheKey, $next);
// Job done, free the lock
$lock->release();

return $response;

You want to avoid what’s known as “dogpiling,” where every retry floods the backend. Instead, gracefully tell the client: “We’re working on it. Try again soon.”

private function waitForCacheLock(string $cacheKey, Request $request)
{
  $tries = 0;
  $sleepLength = 1;

  while ($tries < MAX_LOCK_WAIT_TIME) {
    if (Cache::has($cacheKey)) {
      // Someone finished processing it! Return their response
      return $this->handleCachedResponse($cacheKey, $request->path());
    }

    sleep($sleepLength);
    $tries++;
  }

  // If we're here, we've been waiting too long - give up
  throw new LockWaitExceededException();
}

When Idempotency Goes Wrong: The Spotify Incident

When implementing idempotency, the main risk comes from when functions you believe are idempotent don't actually behave in an idempotent manner. This has historically been especially true when Unicode gets involved!

A few years ago, Spotify ran into a curious problem. Being a global provider, they need to handle usernames in all kinds of languages and character sets (famously, the Unicode snowman ☃ is a valid username). But internally they need a consistent format to store these usernames, avoiding duplication. Their systems used a function to “canonicalise” usernames — that is, to convert user-provided names into a standardised internal format. This included transliteration (turning special characters into ASCII equivalents) and lowercasing. So “BigBird” became “bigbird”, no matter how you typed it.

This function was supposed to be idempotent. You could run it once, or ten times, and get the same result. A feature of idempotent functions is that they generally tend to spread through larger codebases. As they should be benign to run more than once on the same input, developers will often add the idempotent function calls to more places "just to be on the safe side". This can help avoid some weird path bugs, and at worst, should be relatively benign.

Unfortunately in Spotify's case, this approach wasn't quite as benign as hoped - one of their idempotent functions was not quite as advertised.

A clever user discovered that by signing up with a username made entirely of superscript Unicode characters — characters that looked like “BIGBIRD”, but weren’t standard ASCII — they could trick the system. The canonicalisation logic didn’t fully normalise the input the first time, but subsequent passes produced a different result.

That might seem like a quirky bug. But this could be combined with another flaw — where password reset links included the canonicalised username — it allowed for full account takeover. An attacker would create an account with the half-normalised username, generate a "reset password" url, and when clicking on it, their username would be pulled from the url along with the token verification to allow password setting. However, this username would be run through the canonicalisation function again before looking up the user record from the database. "Just incase", right? In this case, the canonicalisation of the half-normalised version ("BIGBIRD", user #456) turned it into something different ("bigbird", user #123), and updated the password for user number 123. The attacker could now log in to their target account at will. Uh oh.

One supposedly idempotent function wasn’t, and Spotify got bitten hard.

The bug was ultimately caused by a python library which struggled with invalid Unicode 3.2, rendering the underlying function not idempotent. It has since been fixed, so don't rush to attempt a take-over on Joe Rogan's account and change the bank account to your own. But it’s a reminder that when we label something as idempotent, it needs to be truly idempotent — not just in happy-path tests, but across all edge cases.

Making It Real: Implementation and Behaviour Design

By now, you understand why idempotency matters and how it works in theory. Let’s talk briefly about the practice — how to get it into your stack without losing your mind.

If you’re building with Laravel, Express, Symfony, or any modern web framework, middleware is your best friend. It sits between your incoming requests and your business logic. That means you don’t have to rewrite all your route handlers — just intercept and manage the few requests that need it.

In Laravel, for instance, you might write a middleware that:

• Checks the HTTP verb. If it’s a GET, for example, you skip idempotency entirely.
• Looks for an Idempotency-Key header.
• Builds a cache key, possibly including user ID or path.
• If a response is cached, return it.
• Otherwise, let the request proceed — and once it’s handled, cache the result.

As we've seen in the middleware example above, it's surprisingly little code. And because it’s middleware, you can easily toggle it on or off per route. Want it just on your /checkout endpoint? No problem.

If you’re using Laravel specifically, I’ve bundled this together into a simple open-source Laravel idempotency package you can install. Even if you're not using Laravel, the code is worth a look, to see exactly how little code is required. It includes locking to handle race conditions and options for customizing your cache strategy, key generation, and duplicate behaviour.

Speaking of which...

To Replay or Not to Replay

When your server receives a repeat request, what should it do?

Most systems, like Stripe or Amazon, simply replay the original response. Same HTTP code, same body, as if the first request had never failed. The second request just repeats up where the first left off.

But not all APIs follow that approach.

GoCardless, for example, will return a 409 Conflict if you repeat a request. Inside the response, they’ll tell you: “This already exists — here’s where to find it.” Your client has to follow up with a GET to retrieve the resource.

It’s slightly more work for API consumers, as we need to add extra fallback and retrieval logic, but it can give you more explicit control over how duplicates are managed.

Which is better? There’s no universal answer. The key is to choose one and document it clearly. Your users — especially internal ones — need to know whether a repeat call will quietly succeed or raise an exception. Ambiguity here creates brittle client code and messy support tickets.

Internal Idempotency: Beyond the API

One of the recurring questions I get is whether idempotency is just for public APIs. Not necessarily!

If you’ve got a queue processing invoices or issuing refunds, you want those operations to be idempotent too. Same core logic applies: check if you've seen this job before, use a deterministic key, store the result if not, replay it if so.

use Jobs\SendPaymentLink;

SendPaymentLink::dispatch($invoiceId);


// Jobs\SendPaymentLink.php
function dispatch(int $invoiceId)
{
  // Use $invoiceId as unique key
  $response = $this->cache->getProcessedResponse($invoiceId);

  if (!$response) {
    // Go process the job and put it in cache
    ...
  }

  ...
}

Whether it's HTTP, a message bus, or a scheduled job at midnight — if a repeat operation could do something bad, it’s a good candidate for idempotency.

Final Thoughts: What Makes This Worth Doing?

At first glance, idempotency might feel like defensive plumbing — the kind of invisible code that’s hard to demo and easy to skip. But the truth is, it’s a core part of building APIs people trust.

It’s what stops flaky Wi-Fi from becoming double payments. It’s what lets mobile apps retry silently, without panic. It’s what keeps infrastructure quirks from leaking into user-facing behaviour.

And most importantly, it’s what makes your API predictable. Predictability is a feature - making life easier for users of our APIs, even when the underlying infrastructure or customers aren't behaving as intended!

Further Reading

• Making retries safe with idempotent APIs (AWS)
• Designing robust and predictable APIs with idempotency (Stripe)
• Creative usernames and Spotify account hijacking (Spotify)
• Laravel Idempotency Package

Watch the session