Idempotency for APIs

#Idempotency-Key#Redis SETNX#Distributed Lock#Double Submit#Stripe Pattern

The 3-Line Engineering Rule

1. THE RULE

Any mutating API (POST/PUT) that performs money transfers, order creation, or external side-effects must be strictly idempotent.

2. WHY IT MATTERS

Network timeouts, user double-clicks, and automated retries will duplicate financial transactions or duplicate physical shipments without idempotency guarantees.

3. PRODUCTION FIX

Accept an `Idempotency-Key` header, acquire an atomic Redis lock (`SETNX` with TTL), check for existing cached responses, and return the identical result without re-executing logic.

Architectural Blueprint & Sequence

Hover over nodes to inspect state transitions

IDEMPOTENCY EXECUTION FLOWSCRIBBLE BLUEPRINT

Atomic Redis state checking, duplicate lock detection, and cached response replaying

Client App

POST /v1/payments (Key: abc-123)

Redis State Lookup

GET idempotency:abc-123

Lock Active

Is PROCESSING?

Return 409 Conflict / Wait

Zero Side-Effects

Is COMPLETED?

Return Cached HTTP 200

Execute Payment

Charge card & write DB

Cache Response

Save payload + TTL: 24h

Execution Sequence & Transitions

Client AppRedis State Lookup[Header: Idempotency-Key]

Redis State LookupIs PROCESSING?[status == PROCESSING]

Redis State LookupIs COMPLETED?[status == COMPLETED]

Redis State LookupExecute Payment[key NOT found (acquire lock)]

Execute PaymentCache Response[success]

Cache ResponseClient App[HTTP 201 Created]

Key Engineering Takeaways

Client generates a unique `Idempotency-Key` (UUIDv4) per logical intent.
Store idempotency states: `PROCESSING` (in-flight lock) and `COMPLETED` (cached response).
If a duplicate request arrives while `PROCESSING`, return HTTP `409 Conflict` or poll wait.
Store the hashed request payload alongside the key to detect payload tampering.

The Problem & Production Reality

In distributed systems, the client never knows with 100% certainty if an unacknowledged HTTP request succeeded on the server. If the connection drops while the server is writing the 200 response, the client has to retry. Idempotency guarantees that executing the same request 1 time or 10 times produces the exact same system state.

Anti-Pattern: Blind Mutation Without Transaction Keys

Processing a payment endpoint without checking for duplicate tokens causes double charges whenever a user clicks 'Pay' twice or mobile network drops.

ANTI-PATTERN EXAMPLE (JAVA)

// ❌ ANTI-PATTERN: Double charge nightmare on retry
@PostMapping("/charges")
public ChargeResponse charge(@RequestBody ChargeRequest request) {
    // If client timed out waiting for response and retried, card charged twice!
    return paymentProcessor.charge(request.getAmount(), request.getCardToken());
}

Production Pattern: Distributed Atomic Idempotency Filter

Use Redis `SET key value NX PX 120000` to atomically claim the request token, process business logic, and cache the serialized HTTP response body for subsequent identical requests.

PRODUCTION-READY CODE (JAVA)

// ✅ PRODUCTION PATTERN: Redis-backed Idempotency Interceptor
@Component
public class IdempotencyHandler {

    private final StringRedisTemplate redisTemplate;

    public Object handleIdempotentCall(String key, Supplier<ResponseEntity<?>> execution) {
        String redisKey = "idempotency:" + key;
        
        // 1. Check if already completed and cached
        String cachedResponse = redisTemplate.opsForValue().get(redisKey + ":result");
        if (cachedResponse != null) {
            log.info("Idempotent hit: replaying cached response for key={}", key);
            return deserialize(cachedResponse);
        }

        // 2. Atomic acquire in-flight lock (120s safety TTL)
        Boolean acquired = redisTemplate.opsForValue().setIfAbsent(redisKey + ":lock", "PROCESSING", Duration.ofSeconds(120));
        if (Boolean.FALSE.equals(acquired)) {
            throw new ResponseStatusException(HttpStatus.CONFLICT, "Concurrent request in progress for this Idempotency-Key.");
        }

        try {
            // 3. Execute actual business transaction
            ResponseEntity<?> result = execution.get();
            
            // 4. Cache response for 24 hours
            redisTemplate.opsForValue().set(redisKey + ":result", serialize(result), Duration.ofHours(24));
            return result;
        } finally {
            redisTemplate.delete(redisKey + ":lock");
        }
    }
}

Trade-off Analysis & Staff Recommendations

Aspect	Pros / Strengths	Cons / Hazards	Staff Recommendation
Redis vs Relational DB Storage	Redis provides sub-millisecond atomic locks and native TTL expiration.	If Redis restarts without persistent AOF, idempotency cache could be lost.	Store the active lock in Redis; write completed idempotency records to PostgreSQL alongside the primary business transaction.
Payload Fingerprinting	Detects when a malicious client reuses an Idempotency-Key with different payload parameters.	Requires computing a SHA-256 hash of the JSON body.	Always hash the request body: if key matches but hash differs, return `422 Unprocessable Entity`.

PR Code Review Checklist (Click to verify)

Is `Idempotency-Key` validated as a valid UUID or alphanumeric string (length <= 64)?

Does the key lock expire automatically with a safety TTL if the worker crashes?

Are concurrent duplicate requests rejected with `409 Conflict`?

Is the payload SHA-256 verified to reject payload tampering?

Previous Best PracticeMinimal & Insightful Logging Next Best Practice Redis Cache When Necessary