API Design

API design is the foundational contract that connects clients (mobile apps, web frontends, IoT devices, third-party partners) with backend microservices and databases. In high-level system design interviews, your API defines the interface through which your system satisfies its functional requirements.

A well-designed API must be:

• Intuitive and Consistent: Developers should predict endpoint behaviors, naming conventions, and parameter formats effortlessly.
• Idempotent and Resilient: Networks fail, requests get retried, and duplicate transactions must not corrupt database state.
• Efficient at Scale: Minimize over-fetching, support high-performance streaming or binary RPCs where appropriate, and protect databases with pagination.
• Secure by Default: Prevent unauthorized object access (BOLA/IDOR), enforce rate limits, and sanitize all user payloads.
• Evolvable: Support non-breaking schema updates and deprecate endpoints gracefully across multi-year client release cycles.

1. Architectural Communication Protocols

Selecting the right communication protocol depends on the client type, latency budget, network constraints, and interaction pattern (request-response vs event streaming).

REST (Representational State Transfer)

REST is an architectural style designed around resources identified by Uniform Resource Identifiers (URIs). It relies on standard HTTP methods to perform operations on those resources and communicates state using representations (typically JSON).

The Six Core Constraints of REST

1. Statelessness: Every request from client to server must contain all information necessary to understand and complete the request. The server must not store client context between requests in server memory.
2. Client-Server Architecture: The user interface concerns are separated from data storage concerns, allowing both to scale independently.
3. Cacheability: Responses must implicitly or explicitly define themselves as cacheable or non-cacheable to prevent clients from retrieving stale data and reduce server load (Cache-Control, ETag).
4. Uniform Interface: A standard, predictable way to interact with resources (standard HTTP methods, resource-based URIs, HATEOAS / self-descriptive messages).
5. Layered System: A client cannot tell whether it is connected directly to the end server, or to an intermediary such as a proxy, API gateway, or CDN.
6. Code on Demand (Optional): Servers can temporarily extend or customize client functionality by transferring executable code (e.g., JavaScript).

Semantic HTTP Methods

REST URI Design Best Practices

• Use nouns, never verbs: Resources are entities. Use /v1/orders, never /v1/createOrder or /v1/getOrders.
• Use plural nouns for collections: /v1/users, /v1/events/{eventId}/tickets.
• Represent hierarchies naturally: /v1/venues/{venueId}/sections/{sectionId}/seats.
• Keep nesting shallow: Avoid nesting beyond two levels (e.g., /v1/venues/{id}/events/{id}/tickets/{id}/scans). Instead, flatten sub-resources with top-level identifiers: /v1/tickets/{ticketId}/scans.
• Use kebab-case or snake_case consistently: /v1/user-profiles or /v1/user_profiles (avoid camelCase in URIs).

GraphQL

Created by Facebook in 2012 and open-sourced in 2015, GraphQL is an API query language and server runtime. Instead of having multiple dedicated endpoints returning fixed JSON shapes, GraphQL exposes a single endpoint (/graphql) accepting declarative queries.

Why GraphQL Was Created: Solving Over-fetching and Under-fetching

• Over-fetching: A mobile client needs only a user's name and avatar_url, but a REST endpoint /v1/users/123 returns 45 fields including address, billing details, and preferences, wasting bandwidth on cellular networks.
• Under-fetching (The N+1 Round-trip Problem): To render a dashboard, a client must make multiple round-trips: GET /v1/users/123, then GET /v1/users/123/orders, then GET /v1/orders/456/items. On high-latency mobile networks, this results in significant rendering delays.

# Client requests exactly what it needs in a single network round-trip
query GetUserProfileWithRecentOrders {
  user(id: "usr_9981") {
    name
    avatarUrl
    orders(limit: 3) {
      id
      totalAmount
      status
      items {
        title
        quantity
      }
    }
  }
}

The Dark Side of GraphQL in Distributed Systems

While GraphQL offers great flexibility for frontend developers, it introduces significant backend complexity:

1. The N+1 Database Query Problem: If a query requests 100 users and each user's orders, naive resolvers execute 1 query for users plus 100 queries for orders. Solution: Use Facebook's DataLoader pattern to batch and cache database lookups.
2. HTTP Caching Challenges: All GraphQL queries typically use POST /graphql with identical URLs. Standard HTTP proxy and CDN caching based on URL paths (Cache-Control, Cloudflare, Akamai) cannot cache GraphQL responses out-of-the-box. Solution: Persisted queries (hashing query strings into GET URLs: /graphql?hash=xyz).
3. Denial of Service (DoS) via Nested Queries: A malicious client can submit deeply recursive queries: user { friends { friends { friends { ... } } } }. Solution: Enforce Query Depth Limiting and Query Cost Analysis at the GraphQL Gateway.

gRPC & Protocol Buffers

gRPC is an open-source, high-performance remote procedure call (RPC) framework developed by Google. It operates over HTTP/2 (or HTTP/3) and uses Protocol Buffers (Protobuf) as its Interface Definition Language (IDL) and binary message serialization format.

Why gRPC Outperforms REST/JSON for Internal Microservices

• Binary Serialization: Protobuf serializes data into compact binary payloads without human-readable field names, resulting in payloads that are 3x to 10x smaller than JSON.
• Zero-Copy & Fast Deserialization: Binary parsing consumes 30% to 60% less CPU than parsing JSON strings.
• HTTP/2 Multiplexing: Multiple requests and responses interleave over a single persistent TCP connection, eliminating head-of-line blocking and connection handshake overhead.
• Strict Type Safety: Client and server stubs are auto-generated from .proto contracts across languages (Java, Go, Python, C++, TypeScript).
• Native Streaming: Supports Unary (1:1), Server Streaming (1:N), Client Streaming (N:1), and Bidirectional Streaming (N:N).

syntax = "proto3";

package payment.v1;

service PaymentService {
  rpc ProcessPayment (PaymentRequest) returns (PaymentResponse);
  rpc StreamPaymentEvents (PaymentStreamRequest) returns (stream PaymentEvent);
}

message PaymentRequest {
  string idempotency_key = 1;
  string account_id = 2;
  int64 amount_cents = 3;
  string currency = 4;
}

message PaymentResponse {
  string transaction_id = 1;
  PaymentStatus status = 2;
  int64 processed_at_epoch_ms = 3;
}

enum PaymentStatus {
  UNKNOWN = 0;
  AUTHORIZED = 1;
  SETTLED = 2;
  DECLINED = 3;
}

WebSockets & Server-Sent Events (SSE)

When systems require real-time push communication from server to client:

• WebSockets: Full-duplex, bidirectional persistent TCP connections initiated via an HTTP 101 Switching Protocols handshake. Ideal for collaborative editing (Google Docs), multiplayer gaming, chat applications (Slack, Discord), and high-frequency trading terminals.
• Server-Sent Events (SSE): Lightweight unidirectional text streaming (Content-Type: text/event-stream) from server to client over standard HTTP. Ideal for stock tickers, LLM token streaming (ChatGPT, Claude), and live sports scores.

Protocol Comparison Matrix

2. Advanced API Design Patterns

The Idempotency-Key Pattern (Stripe / Uber Standard)

In distributed systems, the network is unreliable. A client submits a payment request; the server processes it, but the network drops before the 200 OK response reaches the client. If the client retries, how do you prevent double-charging?

Client                      API Gateway / Server                    Redis (Lock/Cache)             Primary Database
  |                                  |                                       |                             |
  |-- POST /v1/payments ------------>|                                       |                             |
  |   (Idempotency-Key: idemp_123)   |-- SETNX lock:idemp_123 -------------->|                             |
  |                                  |<-- Acquired Lock (OK) ----------------|                             |
  |                                  |-- Check idempotency_records --------->|                             |
  |                                  |   (Record not found: first run)       |                             |
  |                                  |                                       |                             |
  |                                  |-- BEGIN TRANSACTION ----------------------------------------------->|
  |                                  |   INSERT INTO payments (...)                                        |
  |                                  |   INSERT INTO idempotency_records (key, response_body, status_code) |
  |                                  |-- COMMIT ---------------------------------------------------------->|
  |                                  |                                       |                             |
  |                                  |-- Cache result in Redis (TTL: 24h) -->|                             |
  |                                  |-- Release lock ---------------------->|                             |
  |<-- 201 Created (Payment Obj) ----|                                       |                             |
  |                                  |                                       |                             |
  |  [Network drops / Client retry]  |                                       |                             |
  |-- POST /v1/payments ------------>|                                       |                             |
  |   (Idempotency-Key: idemp_123)   |-- Check Redis for key --------------->|                             |
  |                                  |<-- Cached 201 Created Payload --------|                             |
  |<-- 201 Created (Identical Obj) --| (Zero duplicate charge executed!)     |                             |

Implementation Rules for Idempotency

1. Client generates a UUIDv4: e.g., Idempotency-Key: 9b1deb4d-3b7d-4bad-9bdd-2b0d7b3dcb6d.
2. Distributed Mutex Lock: When the request arrives, acquire a short-lived Redis lock (SET lock:{idempotency_key} 1 NX EX 30). If lock acquisition fails, another request with the same key is in-flight: return 409 Conflict or wait and retry.
3. Payload Fingerprinting: Hash the request body (SHA-256(request_body)). If a subsequent request arrives with the same Idempotency-Key but a different payload, return 422 Unprocessable Entity ("Idempotency key reused with different parameters").
4. Atomic Record Persistence: Store the generated response status code and body in the database inside the same transaction as the business operation.
5. Cache Hit Replay: Return the exact same status code and cached response body without re-executing any downstream business logic.

Pagination at Scale: Offset vs Keyset (Cursor)

When querying a collection of millions of records (e.g., social media feeds, transaction histories, audit logs), unconstrained queries crash databases and overwhelm memory.

Offset-Based Pagination (page and limit)

The client passes ?page=5000&limit=20 or ?offset=100000&limit=20.

-- Offset Pagination Query
SELECT * FROM events
ORDER BY created_at DESC
LIMIT 20 OFFSET 100000;

• The Scale Flaw ($O(N)$ Disk Traversal): The database storage engine must read and discard 100,000 index entries before returning 20 rows. At large offsets, queries take seconds and cause database CPU spikes.
• The Data Drift Flaw: If a new row is inserted while the user is reading page 1, row 20 shifts onto page 2. When the user loads page 2, they see the same item again (duplicate read). If an item is deleted, an item gets skipped entirely.

Keyset / Cursor-Based Pagination ($O(1)$ Indexed Seek)

Instead of counting rows, the client supplies an opaque cursor representing the pointer to the last record received.

-- Keyset Pagination Query (Instant B-Tree Seek)
SELECT * FROM events
WHERE (created_at, id) < ('2026-09-13 14:30:00.000', 'evt_9982')
ORDER BY created_at DESC, id DESC
LIMIT 21;

{
  "data": [
    { "id": "evt_9981", "title": "Staff Engineer System Design Bootcamp", "created_at": "2026-09-13T14:28:10Z" },
    { "id": "evt_9980", "title": "Distributed Caching Masterclass", "created_at": "2026-09-13T14:15:00Z" }
  ],
  "pagination": {
    "limit": 20,
    "has_more": true,
    "next_cursor": "eyJjcmVhdGVkX2F0IjoiMjAyNi0wOS0xM1QxNDoxNTowMFoiLCJpZCI6ImV2dF85OTgwIn0=",
    "previous_cursor": null
  }
}

• Constant-time Performance: Uses index seeks (created_at, id) in $O(\log N)$ time regardless of depth.
• Immune to Data Drift: Insertions and deletions do not cause duplicated or skipped records during live scrolling.

Asynchronous Long-Running Jobs (The 202 Accepted Pattern)

Operations that exceed 200–500 milliseconds (e.g., generating 100-page PDF financial reports, ML model training, video transcoding, bulk CSV imports) must never block the client connection. Synchronous connections risk gateway timeouts (504 Gateway Timeout) and tie up web server threads.

The Asynchronous Polling Workflow

1. Initial Submission: Client sends POST /v1/reports.
2. Server Response: Return 202 Accepted immediately with a Location header pointing to the job status resource and a suggested retry interval.

HTTP/1.1 202 Accepted
Location: /v1/reports/jobs/job_8832a
Retry-After: 30
Content-Type: application/json

{
  "job_id": "job_8832a",
  "status": "queued",
  "created_at": "2026-09-13T15:00:00Z",
  "poll_url": "https://api.beingsde.in/v1/reports/jobs/job_8832a"
}

1. Status Polling: Client checks GET /v1/reports/jobs/job_8832a:

{
  "job_id": "job_8832a",
  "status": "completed",
  "progress_percentage": 100,
  "result": {
    "download_url": "https://storage.beingsde.in/exports/fin_report_2026.pdf",
    "expires_at": "2026-09-14T15:00:00Z"
  }
}

Batch and Bulk API Operations

When clients need to create or modify multiple resources at once, issuing hundreds of individual HTTP requests wastes network overhead.

Two Batch Strategies

1. All-or-Nothing Atomic Batch: If any item fails, the entire transaction is rolled back. Returns standard 200 OK on total success or 400 / 422 on validation failure.
2. Partial Success Execution (`207 Multi-Status`): Each item executes independently. The response body maps the status of every item individually.

HTTP/1.1 207 Multi-Status
Content-Type: application/json

{
  "results": [
    { "index": 0, "id": "evt_101", "status": 201, "message": "Created" },
    { "index": 1, "id": "evt_102", "status": 409, "error": { "code": "DUPLICATE_SLUG", "message": "Event slug already exists" } },
    { "index": 2, "id": "evt_103", "status": 201, "message": "Created" }
  ]
}

3. API Versioning Strategies

APIs evolve constantly as business requirements change. You must support multiple client versions without breaking legacy mobile applications that cannot be forced to upgrade immediately.

1. URI Path Versioning (Recommended for Most Systems)

Include the major version in the URL path: https://api.beingsde.in/v1/events vs https://api.beingsde.in/v2/events.

• Pros: Simple, explicit, transparent in logs and analytics, easily routed by API gateways (Kong, Envoy) to separate microservice deployments.
• Cons: Major version upgrades duplicate entire endpoint trees.

2. Header / Content Negotiation Versioning

Pass the version in request headers:

• Custom Header: X-API-Version: 2.1.0
• Accept MIME Header: Accept: application/vnd.beingsde.v2+json
• Pros: Keeps URLs pure and clean.
• Cons: Harder to test and debug directly in web browsers, more complex CDN cache configuration.

3. Query Parameter Versioning

Pass version as a parameter: https://api.beingsde.in/events?version=2 or date-based ?api-version=2026-09-01 (Stripe model).

The Golden Rules of Non-Breaking API Evolution

• Always additive: Adding a new optional field to a response body is NOT a breaking change.
• Never delete or rename existing fields: Deprecate old fields and populate both old and new fields simultaneously for at least 6 to 12 months.
• Use the `Sunset` HTTP Header: Announce retirement dates explicitly: Sunset: Wed, 11 Nov 2026 00:00:00 GMT.

4. API Security, Authentication & Resiliency

Security is not an afterthought; it must be designed into endpoint contracts from day one.

Authentication vs Authorization

• Authentication (AuthN - "Who are you?"): Validating the identity of the caller.
• Authorization (AuthZ - "What are you allowed to do?"): Verifying whether the authenticated entity has permission to perform the requested operation on the target resource.

JWT (JSON Web Tokens) vs API Keys

Mitigating BOLA / IDOR (Broken Object Level Authorization)

The #1 vulnerability on the OWASP API Security Top 10. Consider this vulnerable endpoint: DELETE /v1/bookings/book_9921 If the server checks only that the request has a valid JWT, user Alice could delete user Bob's booking simply by altering the ID in the URL. Mitigation: Every database query must scope the lookup by the authenticated tenant/user ID:

-- Secure AuthZ Query
DELETE FROM bookings 
WHERE id = :booking_id AND user_id = :authenticated_user_id;

Rate Limiting & Throttling Headers

To protect downstream databases and prevent denial-of-service, API gateways enforce rate limiting. Transparent APIs communicate limit thresholds via standard HTTP response headers:

HTTP/1.1 200 OK
X-RateLimit-Limit: 1000
X-RateLimit-Remaining: 984
X-RateLimit-Reset: 1726243200
Content-Type: application/json

When a client breaches rate limits, return 429 Too Many Requests with a Retry-After header indicating how many seconds the client must pause before retrying:

HTTP/1.1 429 Too Many Requests
Retry-After: 60
Content-Type: application/problem+json

{
  "type": "https://api.beingsde.in/errors/rate-limit-exceeded",
  "title": "Too Many Requests",
  "status": 429,
  "detail": "Quota exceeded. Maximum allowed is 1000 requests per hour. Try again in 60 seconds.",
  "instance": "/v1/events/123/bookings"
}

5. Standardized Error Handling (RFC 7807)

Never return unformatted generic 500 error strings or raw SQL stack traces to clients. Use the RFC 7807 Problem Details for HTTP APIs specification:

{
  "type": "https://api.beingsde.in/errors/validation-failed",
  "title": "Unprocessable Entity",
  "status": 422,
  "detail": "Your request parameters failed validation checks.",
  "instance": "/v1/events/evt_101/bookings",
  "code": "BOOKING_VALIDATION_ERROR",
  "timestamp": "2026-09-13T16:22:10Z",
  "request_id": "req_8849b291",
  "invalid_params": [
    {
      "name": "tickets.vip_count",
      "reason": "Requested quantity (5) exceeds max available tickets (2)."
    },
    {
      "name": "payment_method.card_cvv",
      "reason": "CVV must be 3 or 4 digits."
    }
  ]
}

6. The 5-Minute FAANG Interview Action Plan

When your interviewer asks you to design the API for a system (e.g., Ticketmaster, TinyURL, Uber, Twitter):

1. Step 1: Declare the Protocol (30 Seconds)
2. State: *"For external client communication, we will use REST over HTTPS with JSON payloads due to universal client compatibility and CDN cacheability. For inter-service communication between our booking service and payment service, we will use gRPC over HTTP/2 for sub-millisecond binary serialization."*
3. Step 2: Identify Key Resources (1 Minute)
4. Identify 2 to 3 core nouns (e.g., /events, /bookings, /users).
5. Step 3: Define Critical Endpoint Contracts (2 Minutes)
6. Detail request parameters, HTTP verbs, and response codes for the 3 most important operations.
7. *Example*:
8. GET /v1/events?city=NYC&cursor=abc&limit=20 $ ightarrow$ 200 OK
9. POST /v1/events/{id}/bookings (with Idempotency-Key header) $ ightarrow$ 201 Created
10. DELETE /v1/bookings/{id} $ ightarrow$ 204 No Content
11. Step 4: Proactively Call Out Edge Cases & Scale Mechanics (1.5 Minutes)
12. Mention cursor-based pagination to prevent memory bloat on large result sets.
13. Mention the Idempotency-Key header on write endpoints to prevent duplicate charges.
14. Mention rate limiting (429 Too Many Requests) at the API Gateway layer.
15. Mention RBAC / tenant isolation checks to prevent BOLA/IDOR security vulnerabilities.