Idempotency key: 8 rules for safer API retries

An idempotency key identifies one intended operation so an API can recognize repeat attempts. If a customer submits a payment and the connection drops, the client can retry with the same key. A well-designed service can then avoid creating a second payment just because the first response never arrived.

This matters because a timeout does not reveal whether the server completed its work. The request may have failed before reaching the service, or it may have succeeded before the response disappeared. Blindly retrying can duplicate an order, refund, booking, or account change. Refusing every retry creates a different problem: users cannot finish legitimate work.

Idempotency helps resolve this uncertainty, but a header alone cannot guarantee safe behavior. The client, API, database, and downstream systems must agree on what counts as the same operation. This guide explains eight design rules, a payment example, and the checks a team should complete before release.

1. Define the business operation before choosing a key

Start with intent. A customer may buy the same item twice, request two partial refunds, or submit a replacement order after cancelling the first. Those actions can have identical fields while representing different decisions. Therefore, a hash of the request body rarely provides a complete identity policy by itself.

Define the operation in business terms, such as “collect the approved deposit for this checkout attempt.” Then identify who creates its key, where the client stores it, and when that intent ends. A transport retry should reuse the key. A genuinely new purchase should receive a new one.

The AWS Builders’ Library discussion of idempotent APIs explains why client request identifiers help distinguish repeated delivery from repeated intent. This distinction also makes support conversations clearer. Staff can investigate one operation rather than guessing whether similar requests belong together.

Keep identity separate from authorization

A key is not permission to perform an action. Each request still needs authentication, access checks, and tenant boundaries. Otherwise, a caller who learns another customer’s key might gain access to a stored response or interfere with an operation.

2. Create and retain the key before the first attempt

The client should create a sufficiently unpredictable identifier before it sends a request. It must retain that identifier through the period when it may need to retry. If it creates a fresh key after every timeout, the server sees new operations and cannot reliably suppress duplicates.

For a browser checkout, this may require a durable checkout record rather than a value held only in a button handler. For a background job, store the operation identifier with the job. When workers restart or messages arrive again, they can recover the same identity.

Keep sensitive information out of the key. Customer names, email addresses, card details, and access tokens do not improve duplicate detection. They do increase exposure when headers appear in logs or diagnostic tools. Use a separate, access-controlled lookup when support staff need to connect an opaque key to a customer case.

Also scope the key deliberately. A service may combine tenant, endpoint, and operation identifier when it looks up a request. Document that rule so two tenants cannot collide and clients do not assume that a key works across unrelated actions.

3. Reject reuse with different business parameters

A repeated key should carry the same intended request. Suppose a first attempt asks for a refund of 50 units, but a retry asks for 80. Treating both as interchangeable could return misleading success or apply the wrong financial action.

Store a normalized fingerprint of the relevant parameters alongside the key. Compare later attempts with that fingerprint. If the values differ, return a clear conflict or validation response according to the API contract. The client should correct its request or create a new operation after an explicit business decision.

Normalization needs care. JSON field order may not matter, while currency, amount, destination account, and tenant almost certainly do. Similarly, an omitted field and an explicit default may or may not mean the same thing. Define these cases rather than leaving them to accidental serialization behavior.

Document parameter mismatch behavior in your API examples. Then include it in contract testing for API integrations, so clients learn about a change before production retries encounter it.

4. Claim an operation atomically under concurrency

Two copies of a request can arrive almost together. A simple sequence that checks for an existing key and then inserts a result has a race: both workers may observe an empty record and start the action.

Use an atomic claim supported by the storage design, such as an appropriately scoped unique constraint and transaction. One worker becomes responsible for the operation. The other sees a known state and follows the documented duplicate path. Test this with concurrent requests, because a sequential replay test will not expose the race.

Represent at least the states your recovery process needs. These often include in progress, completed, and a form of unresolved or failed execution. Record enough information to distinguish an action that never started from an action whose outcome remains unknown.

An in-progress duplicate needs an explicit response policy. The service might return a conflict with retry guidance, provide an operation status resource, or wait within a bounded period. It should not start another side effect simply because the first worker is slow.

5. Make the result and side effect agree

The difficult failure window lies between performing an action and recording its result. If a service creates an order but crashes before saving its idempotency record, the next attempt may create another order. Conversely, recording success before the order exists can produce a response that describes work that never happened.

When both records live in one transactional database, design the transaction so the business change and operation state commit together where feasible. However, a database transaction cannot automatically include an independent payment provider or shipping service.

For an external call, use the provider’s supported idempotency mechanism, retain its operation reference, and define a reconciliation path. The local workflow must recover after a crash without assuming that an absent local response means the provider did nothing.

Longer processes may need a saga pattern for recoverable business transactions. Each step still needs duplicate handling. Compensation also needs protection, because a repeated refund or stock release can cause damage of its own.

6. Specify what repeated responses mean

Do not assume every provider implements the same policy. Some services replay a stored response; others expose a status resource or retry an incomplete action. Retention periods, validation failures, concurrency conflicts, and server errors can all have different rules.

For example, Stripe’s idempotent request documentation distinguishes behavior across API versions. Its API v1 description explains that it saves the status and body once execution starts, including a server error response. The documentation also describes cases where validation or a concurrent conflict prevents a result from being stored. Read the rules for the exact API version you use.

Your own contract should explain whether a replay returns the original response, a current representation, or a reference to the operation. Include relevant headers and status codes. If a resource changed after the original request, returning its current state has different meaning from replaying the original result.

Keep authorization current even when replaying a response. A stored response must not bypass access restrictions that now apply to the caller.

7. Match retention and retries to real delays

Retention creates a practical boundary. Once a service removes a key, it may treat the same value as new. Therefore, the client’s retry window must fit the server’s documented retention policy. Do not infer an unlimited guarantee from the word idempotent.

Consider delayed queue messages, mobile devices that reconnect much later, manual support replays, and long outages. If any can outlive retention, the business record may need an additional uniqueness rule or reconciliation check. For example, a unique approved deposit reference can provide protection beyond a temporary request cache.

Retries also need limits. Use appropriate backoff, jitter, and a total attempt or time budget. Honor provider guidance where available. Repeating a request continuously can overload a struggling dependency even if every attempt shares one key.

When using a message queue for background work, separate message delivery identity from business operation identity. Two different messages can refer to the same approved action. Conversely, one message may legitimately trigger several distinct operations.

8. Provide a visible path for uncertain outcomes

Some operations remain uncertain after all automatic retries stop. A payment provider may have accepted a charge while its status endpoint is unavailable. The correct response is not to fabricate failure or invite an immediate second payment.

Expose an honest pending state and an operation reference. Tell the customer what will happen next without promising a completion time that the system cannot support. Behind the interface, assign reconciliation work to a scheduled process or a named operational queue.

Record state transitions, attempts, downstream references, and authorized manual decisions. Use audit logging to support investigation while redacting sensitive request data. Logs should help answer whether the action occurred, who resolved it, and why a later attempt was safe.

Manual tools need the same safeguards as public endpoints. If an operator can bypass duplicate checks with a “retry” button, the overall process still permits duplicate effects.

A checkout timeout example

Consider a checkout service that collects a deposit before confirming an order. It creates a local operation with a stable key and the approved amount. The payment request uses that identity through the provider’s supported mechanism. The provider accepts the payment, but the checkout worker loses its connection before receiving the response.

When the customer retries, the checkout service finds the existing operation. It checks that the amount and customer scope match, then follows its recovery path. It may retrieve the provider’s status or repeat the request under the provider’s documented policy. It does not create a new payment identity merely because the browser timed out.

After confirmation, the service records the provider reference and completes the order. If the result remains unclear, it shows pending and schedules reconciliation. This approach cannot remove every outage, but it gives uncertainty a controlled state instead of turning it into another charge.

Release checks that reveal weak designs

Test the same request sequentially and concurrently. Then interrupt execution before the claim, after the business write, and before the response. Include a worker restart, a changed payload, a delayed duplicate, and an expired key. Each test should have a specific expected business outcome.

Check that a second tenant cannot retrieve the first tenant’s result. Verify that logs contain useful references without secret values. Finally, ask a support operator to resolve an uncertain case using the available tools. If they must guess whether an action occurred, improve the evidence before release.

At Oxen Softwares, a sound integration design starts with these business rules and failure paths. Selecting a key format comes later. The goal is a workflow in which customers can retry safely and staff can explain every completed or unresolved action.