Designing Real-Time Systems

# CHAPTER 15

Designing Real-Time Systems #

1. Chapter Introduction #

Designing a standard CRUD application (Create, Read, Update, Delete) using REST APIs is straightforward. The complexity multiplies exponentially when an interviewer asks: "Design WhatsApp," "Design a Live Stock Ticker," or "Design Google Docs Collaborative Editing." These require Real-Time Systems. You cannot expect a user to refresh the page to see a new message. This chapter explores the protocols used to push data from the Server to the Client instantly, comparing Polling, Server-Sent Events, and WebSockets.

2. The Problem with Standard HTTP #

Standard HTTP is a Request-Response protocol. It is unidirectional. The Client must initiate the request. The Server responds, and the connection is closed. *The Limitation:* The Server has no way to say, "Hey Client, your friend just sent you a message!" unless the Client asks first.

3. Attempt 1: Short Polling (The Inefficient Approach) #

*How it works:* The Client sends a standard HTTP request to the Server every 2 seconds: "Any new messages?" *Pros:* Extremely simple to build. *Cons:* It wastes massive amounts of bandwidth and mobile battery. If you have 1 million active users polling every 2 seconds, your server is processing 500,000 requests per second, and 99% of those responses will be "No new messages."

4. Attempt 2: Long Polling (The Workaround) #

*How it works:* The Client sends an HTTP request. Instead of responding immediately, the Server *holds the connection open* until new data is available. Once a message arrives, the Server sends it, closes the connection, and the Client immediately opens a new Long Poll connection. *Pros:* Significantly reduces empty requests compared to Short Polling. *Cons:* Connections eventually timeout. Server memory is consumed holding thousands of idle HTTP connections open.

5. Attempt 3: Server-Sent Events (SSE) #

*How it works:* The Client establishes a persistent HTTP connection. The Server can now continuously push data to the Client over this single connection. *Pros:* Native HTTP. Perfect for unidirectional data streams (e.g., Live Stock Tickers, Live Sports Scores). *Cons:* Unidirectional. The Client cannot use this connection to send data *back* to the Server. (The Client must use standard REST POST requests to send data).

6. The Champion: WebSockets #

WebSockets provide a persistent, full-duplex, bi-directional communication channel over a single TCP connection. *How it works:* The Client starts with a standard HTTP request containing a "Upgrade: websocket" header. The Server agrees, and the HTTP protocol is upgraded to a raw WebSocket connection. Now, the Client and Server can instantly ping messages back and forth indefinitely. *Pros:* Ultra-low latency. True bi-directional real-time communication. *Cons:* Highly complex to scale. Load balancers must be configured to maintain long-lived connections.

7. Scaling WebSockets (The Chat App Challenge) #

*Interview Question:* Design WhatsApp. You chose WebSockets. User A and User B are chatting. *The Problem:* WebSockets are stateful. User A's connection is locked to Server 1. User B's connection is locked to Server 2. If User A sends a message to User B, how does Server 1 send it to Server 2? *The Solution:* A Pub/Sub Message Broker (Redis Pub/Sub or Kafka).

1. 1. User A sends message to Server 1.

1. 2. Server 1 publishes the message to Redis on topic channeluserB.

1. 3. Server 2 (and all other servers) are subscribed to Redis.

1. 4. Server 2 sees the message, realizes User B is connected to it, and pushes the message down User B's open WebSocket.

8. Handling Disconnections (Presence Servers) #

In real-time systems, showing who is "Online" or "Offline" is critical. Because mobile networks are unstable, WebSockets frequently drop. You cannot update the database to "Offline" the millisecond a socket drops (they might drive through a tunnel and reconnect in 2 seconds). *The Fix:* Use a Heartbeat (Ping/Pong) mechanism. The Client pings the server every 5 seconds. A dedicated "Presence Service" updates a Redis cache with the Last Active Timestamp. If a user misses 3 heartbeats (15 seconds), they are marked "Offline."

9. Real-World Scenario: Designing a Live Video Stream (Twitch) #

*Candidate:* "I will use WebSockets to stream the 4K video data." *Interviewer:* "Fail." *The Reality:* WebSockets are designed for lightweight text/JSON messaging (Chat, Notifications). They are terrible for heavy, continuous media streaming. For Live Video, you use specialized protocols like HLS (HTTP Live Streaming) or WebRTC (for ultra-low latency peer-to-peer video calls like Zoom). *The Hybrid approach:* Twitch uses HLS to stream the video player over a CDN, and uses WebSockets specifically for the live text chat box next to the video.

10. Mini Project: Decision Matrix #

Memorize this matrix for your interview:

• Chat Application (WhatsApp) -> WebSockets (Bi-directional)

• Live Stock Ticker / Live Scores -> Server-Sent Events (SSE) (Server -> Client only)

• Real-time Location Tracking (Uber) -> WebSockets (Constant bi-directional updates)

• Video Conferencing (Zoom) -> WebRTC (Peer-to-peer low latency)

11. Common Mistakes #

• Assuming WebSockets replace REST: WebSockets are resource-intensive. Do not use them to fetch a user's static profile data or process a credit card. Only use WebSockets for the specific features that strictly require sub-second real-time interaction.

• Forgetting Load Balancer Limits: Standard load balancers timeout idle connections after 60 seconds. You must explicitly configure the Load Balancer to allow long-lived WebSocket connections, or the LB will kill the chat session.

12. Best Practices #

• Connection Limits: A single Node.js or Go server can handle roughly 10k to 50k concurrent WebSocket connections before running out of file descriptors or RAM. You must explicitly calculate how many chat servers you need based on active users.

13. MCQs #

14. Interview Questions #

• Q: "Design a real-time collaborative code editor. How do you handle conflict resolution if User A and User B type on the exact same line at the exact same millisecond?" (Hint: Look up Operational Transformation / CRDTs).

15. FAQs #

• Q: Are WebSockets secure?

A: Yes, if you use the wss:// (WebSocket Secure) protocol instead of ws://. This encrypts the data using TLS, just like HTTPS.

16. Summary #

Designing real-time systems requires abandoning the standard HTTP Request-Response model. For simple Server-to-Client data feeds, use Server-Sent Events (SSE). For true bi-directional interaction (Chat, Uber tracking), utilize WebSockets. Scaling WebSockets requires careful management of persistent connections and a centralized Redis Pub/Sub layer to route messages between isolated servers. Never use WebSockets for heavy video streaming; rely on WebRTC or HLS.

17. Next Chapter Recommendation #

You now possess all the fundamental building blocks of system design: Scaling, Databases, Caching, Load Balancers, APIs, Microservices, and Real-Time protocols. It is time to put the pieces together. In Chapter 16: Common System Design Problems, we will whiteboard complete architectures for standard interview questions like the URL Shortener and the Chat App.