Design Instagram (Photo Sharing & Activity Feed)

Instagram handles billions of uploaded photos, real-time activity notifications, and personalized timeline generation for over 1 billion monthly active users.

1. System Requirements

Functional Requirements

1. Post Photos & Videos: Users can upload high-resolution photos and video clips with captions and tags.
2. Follow Social Graph: Users can follow/unfollow other accounts.
3. News Feed Generation: Users can view an infinite scroll timeline of recent posts from accounts they follow.
4. Like & Comment: Real-time interactions on posts with instantaneous count updates.
5. Explore & Search: Discover trending posts based on user engagement signals.

Non-Functional Requirements

1. Low Feed Latency: Newsfeed page loads in $< 200\text{ms}$.
2. High Availability: 99.99% availability; read traffic vastly outnumbers writes (100:1 ratio).
3. Media Durability: Zero data loss for uploaded images and videos.
4. Eventual Consistency: A new post does not need to show up on all followers' screens simultaneously, but should appear within seconds.

2. Capacity & Scale Estimation

• Daily Active Users (DAU): $500\text{ Million}$
• New Posts Uploaded per Day: $100\text{ Million}$
• Average Photo Size: 2 MB (compressed to multiple WebP resolutions).
• Daily Media Storage: $100\text{M} \times 2\text{ MB} = 200\text{ TB/day} \rightarrow 73\text{ PB/year}$.
• Average Read QPS: $500\text{M} \times 20\text{ feed views / 86400} \approx 115,000\text{ QPS}$.

3. High-Level Architecture Blueprint

                                  +-----------------------+
                                  |    Mobile Client      |
                                  +-----------+-----------+
                                              |
                                              v
                                  +-----------------------+
                                  |  Cloudflare Edge CDN  |
                                  +-----------+-----------+
                                              |
                                              v
                                  +-----------------------+
                                  |      API Gateway      |
                                  +-----+-----------+-----+
                                        |           |
             +--------------------------+           +--------------------------+
             |                                                                 |
             v                                                                 v
+------------------------+                                         +------------------------+
|   Media Upload Svc     |                                         |   Feed Generation Svc  |
| (Presigned URLs to S3) |                                         | (Hybrid Fanout Engine) |
+-----------+------------+                                         +-----------+------------+
            |                                                                  |
            v                                                                  v
+------------------------+                                         +------------------------+
| Transcoding Pipeline   |                                         |  Redis Timeline Cache  |
| (FFmpeg Worker Cluster)|                                         | (ZSET of Post IDs)     |
+------------------------+                                         +------------------------+

4. Feed Generation: Push vs. Pull vs. Hybrid Fanout

Push Model (Fanout-on-Write)

• When User A posts, the system looks up all followers and inserts the post_id into every follower's Redis timeline.
• Pros: Feed read is instantaneous ($O(1)$ lookup in Redis).
• Cons (The Celebrity Problem): If Cristiano Ronaldo (600M followers) posts, writing 600,000,000 rows melts the message queue.

Pull Model (Fanout-on-Read)

• When a user opens the app, the system queries their follow list, fetches recent posts from all followed creators, and merges them in memory.
• Pros: No write amplification.
• Cons: Feed generation is slow and expensive at read time.

The Hybrid Solution (Production Standard)

• Regular Users ($< 25,000$ followers): Use Fanout-on-Write. Posts are pushed into followers' Redis caches immediately.
• Celebrities / High-Follower Accounts: Use Fanout-on-Read. Their posts are not fanned out. When a follower loads their feed, the system queries the celebrity's recent posts and merges them into the timeline dynamically.

Key Architecture Trade-Offs & Decision Matrix