Chat Messaging

Functional requirements

• Send text messages one-to-one and in groups of up to a few hundred members
• Deliver messages in real time to recipients who are online
• Store messages for offline recipients and deliver them on reconnect
• Show sent, delivered and read receipts for each message
• Show whether a contact is online or when they were last seen
• Sync a conversation's history to a newly signed-in device

Non-functional requirements

• Online-to-online delivery should take well under 500 ms
• Messages within one conversation must appear in the same order for every member
• An acknowledged message must never be lost, even if a server crashes
• Presence may be a few seconds stale; message delivery may not

Scale assumptions

Worked estimates for this brief, not measurements of any real system. Disagree with them out loud — that is the exercise.

Peak multiplier

~4x in evening hours, so ~37,000 messages/second

Storage per day

800 million × ~200 bytes ≈ 160 GB/day before replication

Messages per day

~40 per user, so 800 million (~9,000/second average)

Daily active users

20 million, as a planning assumption

Sockets per gateway

~50,000 each, so ~100 gateway instances at peak

Concurrent connections

~25% online at peak, so ~5 million open sockets

Suggested API surface

• WS/v1/connect

  Persistent socket: send, receive, receipts and presence

• POST/v1/conversations/{id}/messages

  Send a message with a client-generated id, for retries

• GET/v1/conversations/{id}/messages?after=

  Fetch history after a sequence number

• POST/v1/conversations/{id}/read

  Mark read up to a sequence number

Decisions you should be able to defend

• How order is established
• When the sender gets an acknowledgement
• How receipts are stored
• Retries without duplicates