Core Topics

Scalability is a system's ability to handle growing load by adding resources, either by making individual machines bigger (vertical) or by adding more machines (horizontal). Nearly every system design interview hinges on how you scale past a single box.

Load balancers distribute incoming traffic across multiple backend servers to maximize throughput, minimize latency, and tolerate server failures. They are the front door of almost every horizontally scaled system.

The Domain Name System translates human-readable names like api.example.com into IP addresses through a globally distributed, heavily cached hierarchy. It is also a powerful, if blunt, tool for load balancing and regional failover.

A CDN caches content on edge servers close to users, cutting latency from hundreds of milliseconds to tens, absorbing traffic spikes, and shielding origin servers. Cloudflare, CloudFront, Akamai, and Fastly are the canonical providers.

Proxies are intermediaries that sit between clients and servers: forward proxies act on behalf of clients, reverse proxies on behalf of servers, and modern variants like API gateways and service-mesh sidecars centralize cross-cutting concerns.

API design covers the contract between clients and services: the protocol style (REST, GraphQL, gRPC), and the mechanics that make APIs safe and pleasant at scale, versioning, pagination, idempotency, and webhooks.

Rate limiting bounds how many requests a client can make in a window, protecting services from abuse, runaway clients, and overload while enforcing fair use and pricing tiers. The core algorithms are token bucket, leaky bucket, and window counters.

Real-time features, chat, notifications, live dashboards, collaborative editing, need the server to get data to clients as it happens. The main techniques are short polling, long polling, Server-Sent Events, WebSockets, and WebRTC, each with distinct cost and capability profiles.

Latency, throughput, percentiles, and availability targets are the vocabulary for every quantitative claim in a system design interview, and back-of-envelope math with these numbers separates hand-waving from engineering.

Caching stores frequently accessed data in a fast layer (usually memory) to cut latency and shield backing stores from load. It is often the single highest-leverage optimization in a system design interview.

Choosing between relational and non-relational databases is a foundational interview decision. The right answer depends on data shape, query patterns, consistency needs, and scale, not fashion.

Indexes trade extra storage and write cost for dramatically faster reads. Understanding B-trees versus LSM trees and index design (composite, covering) explains most real-world database performance behavior.

Partitioning splits data across machines when one node can no longer hold or serve it. The choice of partition key and strategy determines load balance, query flexibility, and how painful growth becomes.

Replication keeps copies of data on multiple nodes for availability, durability, and read scaling. The core designs are leader-follower, multi-leader, and leaderless quorums, each with distinct consistency and failover behavior.

CAP and PACELC frame the fundamental tradeoffs between consistency, availability, and latency in distributed systems. Knowing the consistency spectrum lets you match guarantees to product requirements instead of over- or under-engineering.

Consistent hashing assigns keys to nodes so that adding or removing a node remaps only a small fraction of keys, instead of nearly all of them. It underpins distributed caches, Dynamo-style databases, and CDN request routing.

Large systems combine block, file, and object storage for bytes at rest, and inverted-index search engines for finding things in them. Knowing which storage tier and which search architecture fits each workload is a recurring interview theme.

Choosing between a single deployable monolith and a fleet of independently deployed services is fundamentally a tradeoff between development simplicity and organizational scalability. Most systems should start as a well-structured monolith and split only when concrete pressure demands it.

Message queues decouple producers from consumers in time, rate, and availability, enabling async processing and load leveling. The core design decisions are the messaging model (point-to-point vs pub/sub), the delivery guarantee, and how ordering and failure handling work.

Event-driven architecture has services communicate by publishing facts about what happened rather than calling each other directly, trading synchronous coupling for eventual consistency. Its key patterns are event notification, event sourcing, CQRS, and the transactional outbox.

Once a business operation spans multiple services or databases, single-node ACID transactions no longer apply. The main tools are two-phase commit (strong but blocking), sagas with compensations (available but eventually consistent), and the supporting machinery of idempotency, distributed locks, and fencing tokens.

Fault tolerance is designing the system to keep serving (possibly in degraded form) when components fail, because at scale something is always failing. The toolkit includes redundancy and failover, circuit breakers, retries with backoff and jitter, bulkheads, graceful degradation, and chaos engineering to verify it all works.

Observability is the ability to ask arbitrary questions about a running system from its outputs, built on three pillars: metrics, logs, and traces. It becomes actionable through methods like RED and USE, SLO-based alerting, and error budgets that turn reliability into a negotiable engineering resource.

System design security covers proving who a caller is (authentication), deciding what they may do (authorization), protecting data in transit and at rest, and defending against the common attack classes. The recurring themes are defense in depth and never trusting the network.

Probabilistic data structures trade exact answers for enormous space savings: approximate set membership, frequency counts, and cardinality in kilobytes instead of gigabytes. Bloom filters, count-min sketch, and HyperLogLog, plus spatial indexes like geohash and quadtrees, appear constantly in real large-scale systems.