Event-Driven Architecture

Events vs Commands

A command is a directed request for someone to do something (ChargeCard, ReserveInventory): it has one intended recipient, the sender cares about the outcome, and it can be rejected. An event is an immutable statement of fact about something that already happened (OrderPlaced, PaymentCaptured): it is past tense, cannot be rejected because it already occurred, and the publisher neither knows nor cares who consumes it. This inversion of knowledge is the whole point: with commands, the sender knows about the receiver; with events, the receiver knows about the sender's events, so adding a new consumer (fraud scoring, analytics, a data warehouse feed) requires zero changes to the producer.

In practice healthy systems mix both. A checkout flow might synchronously command the payment service (the user is waiting and needs a definitive answer), then publish OrderPlaced as an event for everything that can happen eventually: emails, loyalty points, warehouse picking, recommendations. A common smell is commands disguised as events, like SendWelcomeEmailRequested published to a topic with exactly one consumer; that is a command and coupling has just been hidden, not removed.

Event payload design matters too. Thin events (just OrderPlaced plus an order ID) force consumers to call back for details, reintroducing runtime coupling and read load; fat events carry the relevant state snapshot so consumers are self-sufficient, at the cost of larger payloads and schema evolution discipline. Most teams converge on fat domain events with versioned schemas managed through a schema registry.

Event Sourcing

Event sourcing changes the persistence model itself: instead of storing current state and updating it in place, you store the full sequence of events as the source of truth and derive state by replaying them. A bank account is not a row with balance 500; it is AccountOpened, Deposited 300, Deposited 400, Withdrew 200, and the balance is a left fold over that log. Accounting ledgers and git work exactly this way, which is the intuition to offer in interviews.

The benefits are a complete audit trail for free (what was the state on March 3, and why), temporal queries and debugging by replay, and the ability to build entirely new read models retroactively from history, since the events contain everything that ever happened. Replaying millions of events per entity is avoided with periodic snapshots: persist state every N events and replay only the tail.

The costs are real: schema evolution is hard because events are immutable and live forever (you end up with upcasters translating v1 events to v3 on read), querying across entities requires building projections rather than writing a SQL query, deleting data for GDPR requires tricks like crypto-shredding (encrypt per-user, delete the key), and the mental model is unfamiliar to most teams. The honest guidance: event sourcing is excellent for domains that are naturally ledger-like (payments, trading, inventory movements) and overkill as a system-wide default. It also pairs naturally with Kafka-style logs, but a topic with retention is not automatically an event store; an event store needs per-entity streams and optimistic concurrency on append.

CQRS: Separating Reads from Writes

Command Query Responsibility Segregation splits the write model from the read model. Writes go through a model optimized for validating business rules and recording changes; reads are served from one or more projections denormalized for each query pattern. The models are synchronized asynchronously, usually by consuming the write side's events. The motivating fact is that read and write workloads differ wildly: a typical feed-style system might see 100 reads per write, and the shape that makes writes correct (normalized, invariant-enforcing) is the opposite of the shape that makes reads fast (denormalized, precomputed).

A concrete example: an e-commerce order service writes to Postgres, publishes OrderPlaced and OrderShipped events, and projections consume those events to maintain an Elasticsearch index for order search, a Redis view for the user's recent orders, and a warehouse table for analytics. Each read store is disposable and rebuildable by replaying events, which is also the operational escape hatch when a projection has a bug: fix the code, replay, done.

CQRS comes in grades, and saying so shows maturity. Grade one is just separate read and write paths over the same database (different models, maybe read replicas), which most large systems already do. Grade two is separate storage engines updated via events, which buys performance and flexibility at the cost of eventual consistency between write and read sides. Full CQRS plus event sourcing is powerful but should be justified per bounded context, not adopted wholesale. The classic user-facing consequence to design for: a user submits a change, the next page read hits a stale projection, and their edit seems to have vanished. Mitigations include read-your-own-writes (route that user's reads to the write model briefly), returning the updated state in the command response, or optimistic UI updates.

The Outbox Pattern and Eventual Consistency

The dual-write problem is the most common correctness bug in event-driven systems: a service writes to its database and then publishes to Kafka as two separate operations, and a crash between them yields a state change nobody heard about, or an event for a change that rolled back. There is no distributed transaction across a database and a broker, so the fix is the transactional outbox: write the business change and the event into an outbox table in the same local ACID transaction, then a separate relay publishes outbox rows to the broker and marks them sent. The relay is either a poller or, better, change data capture tailing the database's write-ahead log with a tool like Debezium, which is how many teams stream Postgres or MySQL changes into Kafka. Delivery becomes at-least-once, so consumers deduplicate by event ID, which they needed to do anyway.

Eventual consistency is the systemic property you accept in exchange for decoupling: after OrderPlaced is published, there is a window (usually milliseconds to seconds, unbounded during incidents) where inventory, search, and notifications disagree with the order service. Design for it explicitly: make the window observable (consumer lag metrics, end-to-end freshness probes), define per-view staleness budgets (search may lag 30 seconds, the user's own order page may not), and handle the business consequences of the window, for example overselling inventory gets resolved by a compensating cancellation event rather than prevented by a lock.

Two operational realities round this out. First, event ordering and redelivery mean consumers must be idempotent and tolerate out-of-order events across different keys. Second, debugging shifts from reading one stack trace to following a correlation ID across topics, so distributed tracing and a searchable event log are not optional. Interviewers frequently probe exactly here: how do you know the event was published, and what happens if the consumer processes it twice.

Tradeoffs