Interview Case Studies
Design a Real-Time Chat Service
Millions of open connections, messages that must arrive once and in order, and presence that is cheap enough to keep running.
Hard17 min read
By the end of this chapter
- Size and route millions of long-lived connections
- Deliver messages at least once and deduplicate them into exactly-once display
- Order messages per conversation without trusting clocks
- Design presence so its write volume does not dominate the system
Clarify the requirements
- Functional: one-to-one conversations and small groups, up to a few hundred members.
- Functional: messages appear in real time for online recipients and are delivered when offline recipients reconnect, with a push notification meanwhile.
- Functional: every member sees a conversation's messages in the same order.
- Functional: online/last-seen presence and read receipts.
- Out of scope: voice and video calls, very large broadcast channels. End-to-end encryption is worth a mention; it changes what the server can see but not the delivery design.
- Non-functional: low delivery latency for online users, ideally well under a second end to end.
- Non-functional: a message the server has acknowledged is never lost.
- Non-functional: a user with several devices sees the same history on all of them.
Estimate
Sign in to keep a note here. It is free, and nobody else can see it.
Sign inYou’ve reached the end
What this chapter was for:
- Long-lived connections make the gateway tier stateful in a way request/response servers are not; a registry maps users to gateways.
- Delivery is at least once with acknowledgements; a client-generated message id turns duplicates into no-ops.
- Order comes from a per-conversation sequence number assigned by one owner, never from device clocks.
- Presence is high-volume, low-value data: keep it in memory, let it expire, and fan it out lazily.