Hard

Design a Chat System (WhatsApp/Slack)

Design a messaging system supporting one-on-one and group chats with real-time delivery, message ordering, online presence, and read receipts. The core challenges are maintaining millions of long-lived WebSocket connections, routing messages between users connected to different servers, guaranteeing per-conversation ordering, and syncing state across a user's multiple devices.

1Requirements

Functional

  • • One-on-one chat with real-time delivery when both parties are online.
  • • Group chat supporting up to a few hundred members per group.
  • • Offline delivery: messages sent while a recipient is offline are delivered when they reconnect.
  • • Online presence indicators (online, away, last seen).
  • • Sent, delivered, and read receipts per message.
  • • Multi-device support: the same account on phone and desktop sees a consistent history.

Non-functional

  • • Delivery latency under 100 ms between online users in the same region.
  • • At-least-once delivery with client-side deduplication; a message must never be silently lost.
  • • Per-conversation ordering: all participants see messages in the same order.
  • • Scale to 50M concurrent connections and billions of messages per day.
  • • Message history durable and available; support end-to-end encryption as a design consideration.

2Back-of-envelope estimation

Concurrent connections500M DAU, ~10% concurrently connected ≈ 50M WebSocket connections
Message QPS500M DAU x 40 msgs/day = 20B msgs/day ≈ 230K msgs/sec, peak 3x ≈ 700K/sec
Storage per day20B msgs x 100 bytes avg ≈ 2 TB/day, ~730 TB/year
Group amplification1 msg to a 200-member group = up to 200 deliveries; groups multiply delivery traffic ~5-10x
Presence event volume50M users x connect/disconnect + heartbeats every 30s ≈ 1.7M presence events/sec if broadcast naively

3API design

WSS /ws/connect

Upgrade to WebSocket after auth. All real-time traffic (send, receive, ack, typing, presence) flows as frames over this connection. Client heartbeats every ~30s.

POST /api/messages

HTTP fallback to send a message: { conversationId, clientMsgId, content }. clientMsgId makes retries idempotent. Primary path is the WebSocket frame equivalent.

GET /api/conversations/{id}/messages?before={cursor}&limit=50

Fetch message history for a conversation, paginated backwards by (conversation, sequence) cursor.

POST /api/groups

Create a group: { name, memberIds }. Membership changes go through this service and are broadcast as system messages.

POST /api/messages/{id}/receipt

Report delivered/read status; typically sent as a batched WebSocket frame rather than per-message HTTP.

4High-level design

Split stateless HTTP services (auth, profile, group management, history fetch) from stateful chat gateways that hold WebSocket connections. A client first calls a service-discovery endpoint that returns the best gateway (by region and load), then opens a WebSocket and authenticates. Gateways are the only stateful tier; everything behind them scales as ordinary services.

A session registry (Redis) maps user_id → { gateway_id, device_ids }. When Alice sends a message to Bob, her gateway persists the message, looks up Bob's gateway in the registry, and forwards the message to it, which pushes the frame down Bob's socket. If Bob is offline, the message rests in storage and a push notification is triggered; on reconnect Bob's client syncs everything after its last received sequence number.

Message flow is persist-then-deliver: the sender's frame goes to a message service that assigns a per-conversation sequence number, writes to the message store (Cassandra, partitioned by conversation_id, clustered by sequence), acks the sender (single check), then routes to recipient gateways (second check on delivery, blue checks when read receipts come back). Persisting before delivery is what makes at-least-once possible.

Group messages route through the same path with a fan-out step: the message service reads the member list, groups members by their current gateway, and sends one inter-server message per gateway rather than per member, letting each gateway deliver locally to its connected members. Offline members rely on the same sync-on-reconnect mechanism as one-on-one chat.

Presence is its own service: gateways report connect/disconnect and heartbeat timeouts into a presence store, and clients subscribe to presence only for the contacts currently visible on screen, fetched lazily and pushed on change. This turns an O(users x friends) broadcast problem into a bounded pub/sub problem.

5Data model

messages

conversation_id BIGINT PARTITION KEY, seq BIGINT CLUSTERING KEY, message_id UUID, sender_id BIGINT, content BLOB, created_at TIMESTAMP, type TINYINT

Wide-column layout: one partition per conversation, ordered by seq for cheap range scans

conversations

conversation_id BIGINT PK, type TINYINT (dm/group), created_at TIMESTAMP, last_seq BIGINT

group_members

conversation_id BIGINT, user_id BIGINT, role TINYINT, joined_at TIMESTAMP, last_read_seq BIGINT, PK (conversation_id, user_id)

last_read_seq powers read receipts and unread counts

session_registry (Redis)

key user:{id}:sessions → set of { gateway_id, device_id, connected_at }, TTL refreshed by heartbeat

Ephemeral; the source of truth for where to route real-time frames

6Deep dives

WebSockets and the stateful gateway tier

HTTP polling wastes resources and long polling still reopens connections constantly; chat needs a persistent, bidirectional channel, which is exactly what WebSockets provide. Each gateway holds hundreds of thousands of mostly idle connections; the limits are file descriptors, memory per connection (a few KB), and heartbeat processing, not CPU.

Statefulness is the operational cost. You cannot round-robin frames to any server: Bob's frames must reach the specific gateway holding Bob's socket, hence the session registry. Deploys and failures disconnect every client on a gateway, so clients must reconnect with jittered exponential backoff (to avoid a thundering herd) and then run the sync protocol to fetch anything missed while disconnected.

Gateways should do almost nothing: authenticate, maintain heartbeats, forward frames to backend services, and push frames down sockets. All business logic (persistence, sequencing, fan-out) lives in stateless services behind them, so the hard-to-drain stateful tier changes as rarely as possible.

Message ordering and delivery guarantees

Client timestamps cannot order messages: clocks skew, and two messages can carry the same millisecond. Server receive time across multiple servers is also not globally consistent. The robust answer is a per-conversation monotonically increasing sequence number assigned at write time, e.g., by an atomic counter on the conversation's partition owner. Total ordering per conversation is exactly the guarantee users expect, and it doubles as the sync cursor: a client that knows it has everything up to seq 4711 asks for everything after.

Delivery is at-least-once: the sender retries the frame until acked, and the server retries delivery until the recipient acks. Retries create duplicates, so every message carries a client-generated ID (clientMsgId) and receivers deduplicate on it. Exactly-once transport is not achievable in practice; at-least-once plus idempotent receive is the standard pattern.

Gaps are detected by the sequence numbers themselves: if a client holding seq 4711 receives 4713, it knows 4712 is missing and issues a range fetch. This self-healing property is why sequence-based sync beats simply trusting the real-time stream.

Presence and read receipts at scale

Naive presence, broadcasting every connect and disconnect to all friends, generates millions of events per second and mostly updates screens nobody is looking at. Instead: gateways write status changes to a presence store with a TTL refreshed by heartbeat (missed heartbeats flip a user to offline automatically), and clients subscribe only to the presence of users currently rendered (open chat list, active conversation). Flapping connections are smoothed by debouncing: only publish offline if the user stays disconnected for, say, 30 seconds.

Read receipts in one-on-one chat are simple acks flowing back to the sender. In groups, per-message-per-member receipts would be members x messages rows; instead store one last_read_seq per member per conversation, updated as the member reads. "Read by all" for a message is then min(last_read_seq over members) >= message.seq, computed on demand. Receipts should be batched (one frame summarizing many messages) to avoid doubling frame volume.

Multi-device sync and offline delivery

Each device holds its own connection and its own sync cursor (last seq per conversation). Messages route to a user by fanning out to all registered devices; each device acks independently, so the phone being offline never blocks the desktop. Sent messages must also echo to the sender's other devices, which falls out naturally if you treat the sender's other devices as recipients.

Offline delivery is pull-based on reconnect, not a server-side queue replay: because the message store is the queue (partition per conversation ordered by seq), a reconnecting device just asks each conversation for messages after its cursor. This unifies offline delivery, gap repair, and new-device history backfill into one code path.

End-to-end encryption changes the storage contract: the server sees only ciphertext and per-device encrypted copies (each device has its own keys, as in Signal's protocol). Sequencing, receipts, and routing still work since they ride on metadata, but server-side search and history backfill to brand-new devices become client-driven problems.

7Rapid implementation: build the MVP

Theory is table stakes. Here is how you would stand up a working version fast, on a budget, with the core algorithm in real code.

Stack

Node.js + ws (WebSocket) + Redis pub/sub + Postgres on a $12 VPS; two node processes behind nginx to prove cross-server routing

  1. 01Create tables: conversations (id, last_seq BIGINT DEFAULT 0), conversation_members (conversation_id, user_id, last_read_seq), messages (conversation_id, seq, client_msg_id UNIQUE, sender_id, content, created_at, PK (conversation_id, seq)).
  2. 02Stand up a ws server that authenticates a JWT from the connection query string and keeps a Map of userId to socket set.
  3. 03On each server, SUBSCRIBE to a Redis channel per connected user (chan:user:{id}); publish frames there so any server can reach any user.
  4. 04Implement send: assign seq via UPDATE conversations SET last_seq = last_seq + 1 RETURNING last_seq, INSERT the message, ack the sender, then PUBLISH to every member's channel.
  5. 05Make inserts idempotent with the UNIQUE index on client_msg_id: on conflict, re-ack with the existing seq instead of writing a duplicate.
  6. 06Implement sync: on connect the client sends its last seq per conversation and the server returns SELECT ... WHERE seq > $cursor, which also covers offline delivery.
  7. 07Add heartbeats: ping every 30s, terminate dead sockets, and SET presence:{userId} in Redis with a 60s TTL for online indicators.
  8. 08Test by opening two browser tabs pinned to different node processes (nginx ip_hash off) and confirming both directions deliver under 100 ms.

Send path: sequence, persist, then fan out via Redis pub/sub

typescript
async function handleSend(senderId: number, frame: any) {
  const { conversationId, clientMsgId, content } = frame;
  // atomic per-conversation sequence number
  const seqRow = await sql(
    "UPDATE conversations SET last_seq = last_seq + 1 WHERE id = $1 RETURNING last_seq",
    [conversationId]
  );
  const seq = Number(seqRow[0].last_seq);
  try {
    await sql(
      "INSERT INTO messages (conversation_id, seq, client_msg_id, sender_id, content) VALUES ($1,$2,$3,$4,$5)",
      [conversationId, seq, clientMsgId, senderId, content]
    );
  } catch (e: any) {
    if (e.code !== "23505") throw e; // duplicate retry: fall through and re-ack
  }
  const members = await sql(
    "SELECT user_id FROM conversation_members WHERE conversation_id = $1",
    [conversationId]
  );
  const payload = JSON.stringify({ type: "msg", conversationId, seq, senderId, content });
  for (const m of members) {
    pub.publish("chan:user:" + m.user_id, payload); // reaches whichever server holds the socket
  }
  return { type: "ack", clientMsgId, seq };
}

Gateway: socket registry plus per-user Redis subscription

typescript
const sockets = new Map<number, Set<WebSocket>>(); // this server's connections

wss.on("connection", async (ws, req) => {
  const userId = verifyJwt(new URL(req.url!, "http://x").searchParams.get("token"));
  if (!sockets.has(userId)) {
    sockets.set(userId, new Set());
    await sub.subscribe("chan:user:" + userId);
  }
  sockets.get(userId)!.add(ws);
  ws.on("close", async () => {
    const set = sockets.get(userId)!;
    set.delete(ws);
    if (set.size === 0) {
      sockets.delete(userId);
      await sub.unsubscribe("chan:user:" + userId);
    }
  });
});

sub.on("message", (channel: string, payload: string) => {
  const userId = Number(channel.split(":")[2]);
  for (const ws of sockets.get(userId) ?? []) ws.send(payload); // all devices
});

Client sync on reconnect: cursor fetch plus gap detection

typescript
const cursors = new Map<number, number>(); // conversationId -> highest seq seen

async function onReconnect(ws: WebSocket) {
  for (const [conversationId, seq] of cursors) {
    ws.send(JSON.stringify({ type: "sync", conversationId, afterSeq: seq }));
  }
}

function onMessageFrame(msg: { conversationId: number; seq: number }) {
  const have = cursors.get(msg.conversationId) ?? 0;
  if (msg.seq > have + 1) {
    // gap: 4711 -> 4713 means 4712 was missed; fetch the range
    send({ type: "sync", conversationId: msg.conversationId, afterSeq: have });
  }
  cursors.set(msg.conversationId, Math.max(have, msg.seq));
  render(msg);
}

Bottlenecks & failure modes

  • ⚠Gateway restarts disconnect ~200K clients at once; the reconnect stampede needs jittered backoff and connection-rate limiting at discovery.
  • ⚠The session registry is on the path of every message; shard it and cache recent routes on gateways.
  • ⚠Very large or hyperactive groups amplify fan-out; batch per-gateway delivery and consider capping group size or switching huge groups to pull.
  • ⚠A hot conversation partition (a massive group) concentrates writes on one node; sub-partition by seq range or time bucket if needed.
  • ⚠Presence heartbeats at 50M connections are a constant background load; lengthen intervals adaptively and process them off the message path.

Key takeaways

  • ▸Separate the stateful WebSocket gateway tier from stateless logic services; keep gateways dumb and rarely redeployed.
  • ▸Per-conversation sequence numbers solve ordering, gap detection, offline sync, and multi-device cursors with one mechanism.
  • ▸Guarantee at-least-once delivery plus idempotent receive via client message IDs; never promise exactly-once transport.
  • ▸Persist before delivering: the message store, not an in-memory queue, is the source of truth for delivery.
  • ▸Presence must be lazy and subscription-based; broadcasting status changes to all friends does not scale.

Brush up on the underlying topics

Real-Time CommunicationMessage Queues and StreamingSQL vs NoSQLSharding and PartitioningFault Tolerance Patterns