Core Fundamentals• 25 Min
Latency & Scale Numbers Every Architect Must Know
EASY
Latency & Scale Numbers Every Architect Must Know
In system design interviews, demonstrating intuition for physical hardware bounds, I/O latencies, and network transfer speeds separates Junior developers from Staff Engineers.
1. Computer Latency Numbers (Order of Magnitude)
| Operation | Real Time | Scaled to Human Seconds (1 CPU cycle = 1 sec) |
|---|---|---|
| L1 CPU Cache Reference | 0.5 ns | 1 second |
| L2 CPU Cache Reference | 7 ns | 14 seconds |
| RAM (Main Memory) Read | 100 ns | 3.3 minutes |
| Read 1 MB sequentially from RAM | 3,000 ns (3 µs) | 1.6 hours |
| Read 1 MB sequentially from NVMe SSD | 250,000 ns (250 µs) | ~3 days |
| Read 1 MB sequentially from Spinning HDD | 20,000,000 ns (20 ms) | ~7.5 months |
| Same Datacenter Round Trip (LAN) | 500,000 ns (0.5 ms) | ~6 days |
| Cross-Continent Round Trip (SF to NYC) | 40,000,000 ns (40 ms) | ~1.3 years |
| Trans-Atlantic Cable (NYC to London) | 100,000,000 ns (100 ms) | ~3.2 years |
2. Capacity Sizing Rules of Thumb
- Seconds in a Day: $\approx 86,400 \approx 10^5\text{ seconds}$.
- QPS Rule: 1 Million requests per day $\approx 12\text{ QPS}$.
- 100 Million requests per day: $\approx 1,200\text{ QPS}$.
- 1 Billion requests per day: $\approx 12,000\text{ QPS}$.
- Single Redis Instance Max Throughput: 100,000 QPS (single-threaded CPU bound).
- Single PostgreSQL Primary Instance: 5,000–10,000 read QPS, 1,000–2,000 write QPS before sharding.
- Single Kafka Broker Partition: 10–20 MB/sec sequential throughput.
Related Topics
Expand your knowledge by learning about adjacent concepts in system design.
Core FundamentalsMEDIUM
Consistent Hashing Architecture & Implementation
Deep dive into consistent hashing algorithms, virtual nodes, token rings, and minimal key remapping during cluster resizing in distributed databases.
Core FundamentalsMEDIUM
Networking Essentials
Learn the important parts of networking that you'll need to know for your system design interviews
Core FundamentalsMEDIUM
API Design
Master API Design for distributed systems and FAANG interviews: REST vs GraphQL vs gRPC, idempotency keys, cursor pagination, 202 async jobs, security, and RFC 7807 errors.
Cheat Sheet Utility
View and print a concise system design reference card.