Designing an Enterprise Distributed Rate Limiter: Algorithms, Redis Lua, and Edge Enforcement\n\nRate limiters are critical security and reliability boundaries in distributed systems. They protect APIs from abusive spikes, prevent DDoS saturation, enforce tier-based subscription limits, and shield backend databases from cascading thundering herd failures.\n\n---\n\n## 1. Comparing the Core Rate Limiting Algorithms\n\n### 1. Token Bucket\n* How it works: A centralized bucket holds up to $B$ tokens. New tokens are added at a constant rate $R$ tokens/second. When a request arrives, it attempts to consume 1 token. If the bucket is empty, the request is dropped (HTTP 429 Too Many Requests).\n* Pros: Handles legitimate bursts smoothly (can burst up to capacity $B$). Memory-efficient ($O(1)$ state per user).\n* Cons: Parameter tuning ($B$ and $R$) can be delicate.\n* Best For: General REST APIs (e.g., Stripe, GitHub, AWS).\n\n### 2. Leaky Bucket\n* How it works: Requests enter a FIFO queue. Requests are processed and released at a fixed, constant outflow rate.\n* Pros: Completely smooths traffic spikes; provides predictable downstream database load.\n* Cons: Bursts are queued; latency spikes for bursty users.\n* Best For: Asynchronous background jobs and payment gateways.\n\n### 3. Sliding Window Counter\n* How it works: Combines the fixed window counter with the weighted percentage of the previous window:$$\text{Requests} = \text{CurrentWindowCount} + \text{PrevWindowCount} \times (1 - \text{ElapsedTimePercentage})$$\n* Pros: Eliminates boundary burst exploits (where $2 \times \text{limit}$ requests hit right at window edges).\n* Cons: Slight approximation ($~0.05\%$ deviation), but negligible in production.\n* Best For: Enterprise SaaS API quota tracking.\n\n---\n\n## 2. Distributed Race Conditions: Why Plain Redis Gets Broken\n\nA naive rate limiter implementation in Redis uses basic commands:\npython\n# BROKEN DISTRIBUTED IMPLEMENTATION (RACE CONDITION)\ncurrent = redis.get(user_key)\nif current and int(current) > MAX_LIMIT:\n return 429\nredis.incr(user_key)\n\nUnder high concurrency (e.g., 50 concurrent requests hitting 5 application servers at the exact same millisecond), all 50 threads execute redis.get() simultaneously, read the same counter value, and permit all 50 requests through, completely violating the rate limit.\n\n### The Production Fix: Atomic Redis Lua Scripts\nRedis executes Lua scripts atomically in a single thread, guaranteeing no interleaving operations:\n\nlua\n-- ATOMIC SLIDING WINDOW RATE LIMITER LUA SCRIPT\nlocal key = KEYS[1]\nlocal now = tonumber(ARGV[1])\nlocal window = tonumber(ARGV[2])\nlocal limit = tonumber(ARGV[3])\nlocal clearBefore = now - window\n\n-- 1. Remove expired timestamps\nredis.call('ZREMRANGEBYSCORE', key, 0, clearBefore)\n\n-- 2. Count active requests in window\nlocal currentRequests = redis.call('ZCARD', key)\n\nif currentRequests < limit then\n -- 3. Add current request timestamp\n redis.call('ZADD', key, now, now)\n redis.call('EXPIRE', key, math.ceil(window / 1000))\n return 1 -- PERMIT\nelse\n return 0 -- REJECT (429)\nend\n\n\n---\n\n## 3. High-Throughput Optimization: Local Batching & Edge Offloading\n\nIf every single incoming HTTP request requires a network round-trip to Redis, the rate limiter itself can become the primary latency bottleneck:\n\n1. Edge Enforcement (Cloudflare Workers / AWS CloudFront): Block malicious volumetric attacks directly at edge POP locations before traffic touches internal VPCs.\n2. Local Token Reserving (In-Memory Batching): Instead of pinging Redis per request, worker nodes acquire blocks of 50 tokens at a time and deduct locally in RAM. When the local quota nears depletion, a background async worker refills the batch.\n\n---\n## 4. Failure Modes: Soft Fail vs. Hard Fail\n\nWhat happens when the Redis rate limiter cluster crashes or suffers a network partition?\n\n* Fail-Closed (Hard Fail): Block all incoming traffic. Used exclusively for high-risk endpoints (e.g., login passwords, credit card charges).\n* Fail-Open (Soft Fail): Allow requests through and fire critical alerts. Used for consumer applications (e.g., social feed browsing) where uptime is prioritized over strict quota enforcement.\n\n---\n\n## 5. Decision Matrix\n\n| Algorithm | Memory Cost | Handles Bursts? | Boundary Fairness | Best Used In |\n| :--- | :--- | :--- | :--- | :--- |\n| Token Bucket | $O(1)$ | Yes (up to burst size) | High | Public Developer APIs |\n| Leaky Bucket | $O(\text{Queue Size})$ | No (Smooths traffic) | Absolute | Financial Transactions |\n| Sliding Window Counter | $O(1)$ | Moderate | Excellent | SaaS Enterprise Billing |