Glossary & Latency Numbers

Glossary

ACID

The four guarantees of relational database transactions: Atomicity (all or nothing), Consistency (valid state to valid state), Isolation (concurrent transactions do not interfere), and Durability (committed data survives crashes).

Anycast

A routing technique where the same IP address is advertised from multiple locations and BGP delivers each packet to the nearest one. Used by CDNs and DNS resolvers for automatic geographic distribution and failover.

API Gateway

A single entry point in front of backend services that handles routing, authentication, rate limiting, and TLS termination. It hides internal service topology from clients.

Backpressure

A mechanism by which an overloaded downstream component signals upstream producers to slow down, typically via bounded queues or demand signaling. It prevents unbounded buffering and memory exhaustion under load.

BASE

Basically Available, Soft state, Eventually consistent: the loose counterpart to ACID adopted by many distributed NoSQL systems. It prioritizes availability and partition tolerance over immediate consistency.

Bloom Filter

A space-efficient probabilistic data structure that tests set membership with possible false positives but no false negatives. Commonly used to skip expensive lookups, such as avoiding disk reads for keys that definitely do not exist.

Blue-Green Deployment

A release strategy running two identical environments where traffic switches from the old (blue) to the new (green) all at once. Rollback is instant by switching back.

Cache Stampede

A failure mode where a popular cache entry expires and many concurrent requests all hit the backing store to regenerate it simultaneously. Mitigated by request coalescing, staggered TTLs, and serving stale data during refresh.

Canary Release

A deployment strategy that routes a small percentage of traffic to a new version, monitors error rates and latency, and gradually increases the share. It limits the blast radius of a bad release.

CAP Theorem

The principle that during a network partition a distributed system must choose between consistency and availability. Since partitions are unavoidable, systems are characterized as CP or AP by their behavior when one occurs.

CDC (Change Data Capture)

A technique that streams a database's committed changes, usually by reading its write-ahead log or binlog, to downstream consumers as an ordered event feed. Used to sync caches, search indexes, and warehouses without dual writes.

CDN (Content Delivery Network)

A geographically distributed network of edge servers that caches content close to users, reducing latency and offloading the origin. Primarily used for static assets, images, and video.

Checksum

A small value computed from data, such as a hash, used to detect corruption during storage or transmission. Receivers recompute it and compare to verify integrity.

Circuit Breaker

A resilience pattern that trips open after repeated failures to a dependency, failing fast instead of piling up requests, then probes with trial requests before closing again. It prevents cascading failures.

Cold Start

The extra latency incurred when a serverless function or service instance must be initialized from scratch before handling its first request. Mitigated by provisioned concurrency, warm pools, and lighter runtimes.

Compaction

The background process in LSM-based storage engines that merges sorted on-disk segments, discarding overwritten and deleted entries. It reclaims space and keeps read performance bounded at the cost of write amplification.

Connection Pooling

Reusing a set of pre-established connections (typically to a database) instead of opening a new one per request. It avoids handshake overhead and caps the number of concurrent connections the backend must handle.

Consistent Hashing

A hashing scheme that maps nodes and keys onto a ring so that adding or removing a node only remaps about 1/N of the keys. Virtual nodes are used to balance load; it is standard in distributed caches and databases.

CQRS (Command Query Responsibility Segregation)

An architecture that separates the write model from the read model, letting each use its own schema, storage, and scaling, typically synchronized through events. Useful when read and write workloads differ sharply.

DAU (Daily Active Users)

The number of unique users who engage with a product in a day, a standard input for capacity estimation. Interviewers often give DAU and expect derived request rates and storage needs.

Dead Letter Queue

A queue that receives messages that could not be processed after repeated attempts, isolating poison messages so they do not block the main queue. Operators inspect and replay or discard them.

Denormalization

Deliberately duplicating data across tables or documents to avoid joins and speed up reads. It trades storage and write-path complexity for read performance.

Edge Computing

Running compute at locations geographically close to users, such as CDN points of presence, rather than in a central region. It reduces round-trip latency for logic like personalization, auth, and A/B routing.

Eventual Consistency

A consistency model guaranteeing that, absent new writes, all replicas converge to the same value over time, so reads may temporarily return stale data. It enables high availability and low latency in replicated systems.

Exponential Backoff

A retry strategy where the wait between attempts grows exponentially, reducing pressure on a struggling service. Combined with jitter to prevent synchronized retry waves.

Failover

The process of shifting traffic from a failed component to a healthy standby, either automatically or manually. Key metrics are detection time, promotion time, and whether any acknowledged writes are lost.

Fan-out

Distributing one event or request to many recipients, such as delivering a post to every follower's feed. Fan-out on write precomputes results at publish time; fan-out on read assembles them at query time.

Geohash

An encoding that converts latitude and longitude into a short string where shared prefixes indicate spatial proximity. Used to index and shard location data for nearby-search queries.

Gossip Protocol

A decentralized communication pattern where each node periodically exchanges state with a few random peers, spreading information epidemically. Used for membership, failure detection, and metadata dissemination in systems like Cassandra.

Graceful Degradation

Designing a system to keep serving its core function with reduced quality when dependencies fail, using fallbacks, feature flags, and load shedding rather than failing entirely.

Heartbeat

A periodic signal a node sends to indicate it is alive. Missing heartbeats past a timeout trigger failure detection, failover, or leader election.

Hinted Handoff

A technique in leaderless replication where, if a replica is down, another node temporarily accepts its writes along with a hint, then replays them when the replica recovers. It preserves write availability during transient failures.

Hot Spot

A shard, partition, or key that receives disproportionate traffic, overwhelming its node while others sit idle. Classic causes are celebrity users and monotonically increasing keys like timestamps.

Idempotency

The property that performing an operation multiple times has the same effect as performing it once. Essential for safe retries; commonly implemented with client-supplied idempotency keys.

Jitter

Randomness added to retry delays or scheduled intervals so many clients do not act in synchronized waves. It smooths load spikes caused by correlated timing.

Leader Election

The process by which nodes in a distributed system agree on a single coordinator, typically via consensus protocols like Raft or coordination services like ZooKeeper or etcd. Majority quorums prevent two simultaneous leaders.

Linearizability

The strongest single-object consistency model: every operation appears to take effect atomically at some instant between its start and completion, so reads always reflect the most recent write. Also called strong consistency.

Load Balancer

A component that distributes incoming traffic across a pool of servers using algorithms like round robin, least connections, or hashing. Operates at L4 (transport) or L7 (application) and performs health checks to route around failures.

Load Shedding

Deliberately rejecting or deprioritizing some requests when a system nears overload so that remaining requests can be served correctly. Preferable to accepting all traffic and failing everything.

LSM Tree (Log-Structured Merge Tree)

A write-optimized storage structure that buffers writes in memory and flushes them as sorted immutable segments merged by background compaction. Powers Cassandra, RocksDB, and LevelDB; fast writes at the cost of read and write amplification.

Merkle Tree

A tree of hashes where each parent hashes its children, letting two replicas compare roots and descend only into differing branches. Enables efficient anti-entropy synchronization in systems like DynamoDB and Cassandra.

Message Queue

A buffer that decouples producers from consumers, absorbing bursts and enabling asynchronous processing with retries. Each message is typically consumed by exactly one worker in a competing consumer pool.

MTTR (Mean Time To Recovery)

The average time from failure detection to restored service. Modern reliability practice favors minimizing MTTR through fast detection and rollback over maximizing time between failures.

N+1 Query Problem

An access pattern where fetching a list requires one query for the list plus one additional query per item, multiplying database load. Fixed with joins, batch fetching, or data loaders.

Outbox Pattern

A pattern that writes events into an outbox table within the same database transaction as the state change, with a separate relay publishing them to a message broker. It guarantees events are published if and only if the transaction committed.

PACELC

An extension of CAP: if a Partition occurs, trade Availability versus Consistency; Else, in normal operation, trade Latency versus Consistency. It captures the everyday cost of synchronous replication.

Partition Tolerance

A system's ability to continue operating when network failures split nodes into groups that cannot communicate. In practice it is mandatory, forcing the CAP choice between consistency and availability.

Quorum

The minimum number of nodes that must agree for an operation to proceed, usually a majority. With N replicas, requiring W write acks and R read responses where R + W > N ensures reads overlap the latest write.

Rate Limiting

Restricting how many requests a client can make in a time window to protect services from abuse and overload. Common algorithms include token bucket, leaky bucket, and sliding window counters.

Read Replica

A copy of a database that receives replicated writes from the primary and serves read traffic, scaling reads horizontally. Asynchronous replication means replicas can lag and serve slightly stale data.

Replication Lag

The delay between a write committing on the primary and appearing on a replica. It causes anomalies like a user not seeing their own write, mitigated by read-your-writes routing or synchronous replication.

Saga

A pattern for distributed transactions that executes a sequence of local transactions across services, undoing completed steps with compensating transactions if a later step fails. Implemented via choreography (events) or orchestration (a coordinator).

Serializability

The strongest transaction isolation level, guaranteeing that concurrent transactions produce the same result as some serial execution. It eliminates anomalies like write skew at the cost of throughput.

Service Discovery

The mechanism by which services find the current network locations of other services, via a registry like Consul, etcd, or DNS. Essential in dynamic environments where instances scale and move constantly.

Service Mesh

An infrastructure layer, typically sidecar proxies like Envoy managed by a control plane like Istio, that handles service-to-service traffic: mutual TLS, retries, timeouts, load balancing, and observability, without application code changes.

Sharding

Splitting a dataset horizontally across multiple nodes, each holding a subset of rows determined by a shard key. It scales storage and throughput beyond one machine but complicates cross-shard queries and transactions.

Sidecar

A helper process deployed alongside an application container to provide cross-cutting capabilities like proxying, logging, or configuration. The building block of service meshes.

SLA (Service Level Agreement)

A contractual commitment to customers about service performance, such as uptime, with defined penalties for breaches. Usually looser than the internal SLO that backs it.

SLI (Service Level Indicator)

A quantitative measurement of service behavior, such as p99 latency, error rate, or availability. SLIs are the raw signals against which SLOs are set.

SLO (Service Level Objective)

An internal target for an SLI, such as 99.9% of requests succeeding within 200ms over 30 days. The remaining allowance defines the error budget that gates release velocity.

Snapshot Isolation

A transaction isolation level where each transaction reads from a consistent snapshot of the database taken at its start, implemented via MVCC. It avoids most anomalies without read locks but permits write skew.

Split Brain

A failure mode where a partition leaves two nodes both acting as leader, accepting conflicting writes. Prevented by majority quorums and fencing tokens.

Sticky Session

Load balancer behavior that routes all of a client's requests to the same backend instance, usually via a cookie or IP hash. Needed for in-memory session state but hinders even load distribution and failover.

Throughput

The amount of work a system completes per unit of time, such as requests or bytes per second. Often traded against latency, for example through batching.

Thundering Herd

Many clients or processes waking or retrying simultaneously and overwhelming a shared resource, such as after a cache expiry or a service recovering from an outage. Mitigated by jitter, request coalescing, and gradual ramp-up.

Tombstone

A marker written to record a deletion in systems with immutable or replicated storage, so the delete propagates to all replicas before the data is physically removed during compaction.

TTL (Time To Live)

An expiry duration attached to cached entries, DNS records, or messages, after which they are discarded or refreshed. TTLs bound staleness and enable automatic cleanup.

Two-Phase Commit (2PC)

An atomic commit protocol where a coordinator first asks all participants to prepare, then instructs all to commit or abort. It guarantees atomicity across nodes but blocks if the coordinator fails after prepare, hurting availability.

Vector Clock

A logical clock assigning each node a counter vector to track causality between events without synchronized time. It distinguishes ordered updates from concurrent conflicting ones in leaderless replication.

WAL (Write-Ahead Log)

An append-only log where changes are durably recorded before being applied to the main data structures, enabling crash recovery by replay. Also the foundation of replication streams and CDC.

WebSocket

A protocol providing a persistent, full-duplex connection between client and server over a single TCP connection, upgraded from HTTP. Used for chat, live collaboration, and gaming where both sides push data.

Write Amplification

The ratio of bytes physically written to storage versus bytes logically written by the application, caused by compaction, page rewrites, or SSD garbage collection. High amplification wears SSDs and consumes I/O bandwidth.

Zero Downtime Deployment

Releasing new software without interrupting service, using strategies like rolling updates, blue-green switches, or canaries. Requires backward-compatible database migrations and connection draining.