Chat System System Design Interview: Scope, Sketch, Explain

A chat system system design interview is usually won or lost before you draw anything. The interviewer is not grading whether you know the word WebSocket. They are watching whether you can turn a vague prompt into an agreed scope, sketch something small that works, and then explain out loud what you are giving up with each choice. This walkthrough takes you through the whole conversation in order, with the words you might actually say.

Why Chat Systems Show Up So Often in System Design Interviews

Chat is a familiar product. Everyone in the room has used one, so the interviewer does not have to spend ten minutes explaining the problem. That frees the whole session for architecture.

It also touches almost every core topic at once: real-time delivery, storage, per-conversation ordering, presence, scale, and failure handling. One prompt, many threads to pull.

The trap is that familiarity feels like permission to skip ahead. Candidates jump straight to "we'll use WebSockets and a messages table" and then spend thirty minutes discovering that they never agreed on what a message is, how many people receive it, or how long it must live.

In the first thirty seconds, say what you are about to do: "I'd like to agree the scope first, then sketch a simple design, then talk through trade-offs." That one sentence tells the interviewer you have a plan and gives them a chance to redirect you early.

Step 1: Clarify System Design Requirements Before You Draw Anything

Treat this phase as a contract. You are not stalling; you are buying yourself a shared definition of done. For more on how requirements drive a design, see Database Sharding System Design Interview: Lead With the Why and Rate Limiter System Design Interview: Reasoning Out Loud.

Ask these questions, in roughly this order.

Scale and shape. One-to-one only, or group chats? If groups, what is a realistic maximum size? Roughly how many concurrent users are we designing for?
Message types. Text only, or media, reactions, edits, and deletes? Each one changes storage and delivery.
Delivery guarantees. Is at-least-once acceptable? What does "sent" mean to the user — accepted by the server, or delivered to the recipient's device?
History. How far back must messages be readable, and who can read them? A week of history is a very different system from forever.
Ordering. Per-conversation ordering is usually the real requirement. Global ordering almost never is.

Then write the agreed scope back in a few bullets. Here is what a finished scope sounds like — swap in whatever the interviewer states:

"So to confirm: one-to-one plus group chats up to 500 people, text and images, at-least-once delivery, per-conversation ordering, history kept for two years, and online delivery within about a second. Does that match what you have in mind?"

Say it out loud and ask if it matches. You now have a contract you can point back to every time you make a decision.

A short script for the first three minutes

"Before I design anything, can I pin down four things? First, is this one-to-one, or do we need group chat, and what's the biggest group? Second, text only or media too? Third, when I send a message, is it enough that the server has it, or does the user expect it to reach the recipient's device? And fourth, how far back does history go?"

That is thirty seconds of talking. It saves you twenty minutes of guessing.

Step 2: Sketch the Chat System Architecture at a High Level

Start with the obvious pieces, and say that you are starting with the obvious pieces. Clients, a connection layer, a message service, a store, and a notification path.

Explain the two connection options. Long polling has the client ask repeatedly for new data; it works everywhere and costs you latency and wasted requests. Persistent connections keep a channel open so the server can push. For chat, persistent connections are a common choice because the core interaction is the server telling you something happened — but say why: if the client environment cannot hold a connection open, long polling is a reasonable fallback rather than a primary design.

Then describe the write path in one breath:

The client sends a message over its open connection.
The gateway authenticates the session and forwards the message.
The message service assigns an ID and a timestamp and validates the sender's membership in the conversation.
The store persists it.
Fan-out delivers it to the other participants.

And the read path: history comes from storage, live messages come from the connection layer. Two paths, one conversation.

Keep the first diagram deliberately small — five or six boxes. Complexity should arrive only when a requirement forces it. If you draw twenty boxes in minute five, you have nothing left to add when the interviewer says "now make it multi-region." The Rate Limiter System Design Interview: Reasoning Out Loud shows how a small first sketch leaves room to grow.

Step 3: Decide How Messages Fan Out to Recipients

This is the decision the interview usually turns on, and it is where a chat system system design interview rewards comparison over confidence. Compare three options rather than announcing one.

Push on write. When a message arrives, the server immediately delivers it to every recipient's connection. Simple, low latency, and natural for small groups. It gets expensive fast as group size grows, because one send becomes many writes.

Pull on read. Store the message once and let clients fetch what they have not seen. Cheap to write, but you pay in read latency and in the complexity of tracking what each client has already received.

Hybrid. Push to users who are currently online, and let offline users pull on reconnect. This is a common landing point in real chat systems, and it is worth saying so — though it is a design choice, not a universal rule.

Then talk about the inbox-per-user model: each recipient gets a copy or a pointer in their own inbox. It makes reads trivial and writes expensive, and it is the classic fan-out trade-off in a different costume.

Finally, separate presence and typing indicators from the message path. They are low-durability, high-frequency signals. If a typing indicator is lost, nobody cares. If a message is lost, somebody does. Do not let them share a pipeline.

Tie every choice back to a requirement you agreed on earlier. "We said groups up to the maximum we agreed, so push-on-write is still affordable — but if that number grows, this is the first thing I would revisit."

Step 4: Storage, Ordering, and Message Identity

Message history is append-heavy and read by conversation. That shape often fits a wide-column or append-oriented store better than a relational table at scale, because you are writing constantly and reading in time-ordered ranges — but that is a trade-off, not a rule. If your access patterns are simple and your scale is modest, a relational store with the right indexes can be the simpler, cheaper choice.

Ordering is the part candidates hand-wave. You do not need a global clock. You need a per-conversation sequence number, assigned by a single writer for that conversation, so every participant sees the same order. Say that sentence out loud; it is the whole answer.

Identity and idempotency go together. Let the client generate a message ID, and have the server deduplicate on it. Now a retry after a dropped connection does not create a duplicate message. Without this, your at-least-once delivery guarantee becomes visible to users as double texts.

Retention and deletion deserve a sentence each. If a user deletes a message, is it removed for them, for everyone, or tombstoned? If an account is deleted, what happens to the other side of those conversations? You do not need a legal answer, but you do need to show you thought about it.

Read receipts and delivery states are separate records. They are small, they change often, and they are eventually consistent. Keeping them out of the message table keeps your hot write path clean.

Step 5: Reliability, Scale, and Failure Modes

Interviewers often add pressure here. Have a few failure modes ready.

Reconnection storms. A network blip drops thousands of clients at once, and they all reconnect immediately. Exponential backoff with jitter spreads the load. Without jitter, they reconnect in lockstep.

Gateway death. If a connection gateway dies, clients need to resume sessions and replay anything they missed. This is why the offset or sequence number each client has seen matters.

Hot partitions. One very active group chat can overwhelm a single partition. Sharding by conversation helps until one conversation is the hot spot; then you split that conversation's writes or accept a ceiling.

Backpressure. A slow consumer cannot keep up with fan-out. You need a bounded queue and a policy: drop, buffer, or disconnect and let them pull.

Multi-region. Decide where messages are written and how reads stay local. Cross-region writes are slow, so many designs pin a conversation to a home region and accept the read cost elsewhere.

Step 6: Name the System Design Trade-Offs Out Loud

The design is not the deliverable. The reasoning is. Practise the sentence: "I would choose X because of Y, and here is what I give up." A chat system system design interview is mostly a sequence of these sentences, so make each one do real work.

Here is one fully worked, using the latency-versus-durability choice:

"I would acknowledge the send as soon as the message service has assigned an ID and queued it for persistence, because the user experience of a chat is that sending feels instant, and a spinner on every message would feel broken. What I give up is the guarantee that a message survives a crash in the moment between acknowledgement and write. To limit that, I'd keep the queue durable and replay it on restart, and I'd accept that a rare message might arrive late rather than never."

That is the shape to copy. State the choice, the requirement it serves, the cost, and the mitigation. Then do the same for consistency versus availability on read receipts, simplicity versus cost on fan-out, and storage cost versus user experience on long history. Say each one before you are asked, so the reasoning is on the table rather than waiting to be pulled out of you.

Step 7: A Practice Loop for the Chat System Design Interview

Treat rehearsal as a loop, not a performance. Pick one phase — requirements, the diagram, or trade-offs — and run it on its own.

Run the phase once, out loud, recorded. Say it the way you would say it in the room.
Listen back against a short checklist. Two or three items only. For requirements, one item might be: "Did I state my assumptions before asking questions?"
Pick exactly one fix. Not five. One.
Rerun the same phase with that fix. Then move to the next phase on another day.

Here is what the difference sounds like. Weak narration: "I'll use WebSockets and a messages table." Strong narration: "I'm assuming one-to-one plus small groups, text only, and per-conversation ordering. Given that, I'd start with persistent connections and a single message store, and I'd revisit the store if history retention turns out to be long." The second version states assumptions, makes a choice, and names what would change it.

After each loop, write down the one question you wish you had asked, and open the next run with it. More walkthroughs and practice notes are collected on the Blog.

Common Mistakes in a Chat System Design Interview

Designing for far more users than the prompt implies, before agreeing on the actual scale.
Treating WebSockets as the whole answer and skipping storage and ordering.
Ignoring offline delivery, which is where most real chat complexity lives.
Never revisiting the requirements when the interviewer adds a constraint.
Presenting one option as obviously correct instead of comparing alternatives.

Before Your Next Chat System Design Interview

Agree the scope before you draw, and say the trade-off sentence out loud every time you make a choice. Next, record one run of just the requirements phase and check that your assumptions came before your questions.

Frequently asked questions

How much time should I spend on requirements in a chat system system design interview?

Aim for roughly the first five to eight minutes, then move to the diagram. A useful signal that you have gone too long: you are asking questions whose answers would not change a single box or choice. If the interviewer starts repeating themselves or nudging you forward, stop asking and start sketching.

Should I use WebSockets or long polling when designing a chat system?

Name both, then commit in one sentence: "I'd use persistent connections because the server needs to push, with long polling as the fallback if a client cannot hold a connection open." That phrasing shows you know the alternative and the condition that would make you switch, without spending minutes on it.

How do I handle message ordering across multiple servers?

Route all writes for one conversation to the same server, and let that server stamp each message with the next number in that conversation's sequence. Readers then sort by that number. The hard part is what happens when that server fails: you need a clear story for electing a new writer and continuing the sequence without gaps or repeats.

What is the difference between fan-out on write and fan-out on read for chat?

Write fan-out is simple and gives recipients low latency, because the message is copied out as soon as it arrives, but it gets expensive as group size grows. Read fan-out is cheap to write and pays in read latency, since each reader assembles the conversation when they open it. Say which cost your agreed requirements make more acceptable.

Do I need to design group chat if the interviewer only asks for one-to-one messaging?

Design what was asked for, then say what would change if groups were added. Group size is the variable that forces a different fan-out choice. If the interviewer adds groups mid-session, you can say: "That changes my fan-out decision, because one send now becomes many deliveries, so I'd move toward a hybrid model and revisit the store." That shows you can adapt a design under a new constraint instead of starting over.

How can I practise a system design walkthrough without a study partner?

Write out a transcript of one run and read it as if a colleague had sent it to you. Mark every place where a box appears before its reason, and every assumption you made silently. A written transcript is easier to audit than a recording, because you can see the order of your sentences and catch reasoning that only sounded fine in the moment.

Practise the walkthrough and review how clearly you explained each decision.