Design a Unique ID Generator (Snowflake)

1Requirements

2Back-of-envelope estimation

3API design

4High-level design

The Snowflake layout packs four fields into one 64-bit integer: 1 unused sign bit, 41 bits of milliseconds since a custom epoch, 5 bits of datacenter ID, 5 bits of machine ID, and 12 bits of per-millisecond sequence. Because the timestamp occupies the high bits, IDs sort by creation time, which keeps B-tree inserts append-friendly and lets you paginate by ID instead of a separate created_at column.

Each application server runs the generator as an in-process library. On startup it acquires a unique worker ID (datacenter + machine) from a small coordination store such as ZooKeeper, etcd, or even a database table with a unique constraint. After that, ID generation requires no network calls at all: the node reads its local clock, increments a sequence counter, and bit-shifts the fields together.

Within one millisecond, the 12-bit sequence counter distinguishes IDs. If a node exhausts 4,096 IDs in a single millisecond, it spins until the next millisecond tick. Across milliseconds the counter resets to zero. Across machines, uniqueness comes from the worker ID bits, so two machines can never collide even at the exact same timestamp and sequence.

Alternatives worth naming in an interview: UUIDv4 is coordination-free and simple but is 128 bits, random (terrible for B-tree locality), and not time-sortable; UUIDv7 fixes sortability but is still 16 bytes. A database ticket server (Flickr style) uses REPLACE INTO on an auto-increment table, which is simple but adds a network hop and a single point of failure, usually mitigated by two servers handing out odd and even IDs. Snowflake is the sweet spot when you need compact, sortable, high-throughput IDs.

5Data model

6Deep dives

7Rapid implementation: build the MVP

Theory is table stakes. Here is how you would stand up a working version fast, on a budget, with the core algorithm in real code.

1. 01Pick a custom epoch (e.g. 2024-01-01T00:00:00Z) and define the bit layout as constants: 41 timestamp, 10 worker, 12 sequence.
2. 02Write the generator class with BigInt bit-shifting and a lastTimestamp guard that throws on clock regression.
3. 03Create a worker_leases table in Postgres with a UNIQUE(worker_id) constraint; on boot, INSERT the first free ID in 0..1023 with a leased_until timestamp.
4. 04Add a renewal loop that extends the lease every 30 seconds and kills the process if renewal fails twice.
5. 05Wrap the generator in a GET /v1/id route that returns the ID as a string, plus a /decode route for debugging.
6. 06Load test with autocannon at 50k req/sec and verify zero duplicates by inserting all IDs into a UNIQUE column.
7. 07Add a unit test that mocks Date.now going backwards and asserts the generator throws instead of duplicating.

const EPOCH = 1704067200000n; // 2024-01-01 UTC
const WORKER_BITS = 10n;
const SEQ_BITS = 12n;
const MAX_SEQ = (1n << SEQ_BITS) - 1n; // 4095

export class Snowflake {
  private lastTs = -1n;
  private seq = 0n;
  constructor(private workerId: bigint) {
    if (workerId < 0n || workerId > 1023n) throw new Error("worker id out of range");
  }
  next(): bigint {
    let ts = BigInt(Date.now());
    if (ts < this.lastTs) throw new Error("clock moved backwards");
    if (ts === this.lastTs) {
      this.seq = (this.seq + 1n) & MAX_SEQ;
      if (this.seq === 0n) {
        while (ts <= this.lastTs) ts = BigInt(Date.now()); // spin to next ms
      }
    } else {
      this.seq = 0n;
    }
    this.lastTs = ts;
    return ((ts - EPOCH) << (WORKER_BITS + SEQ_BITS)) | (this.workerId << SEQ_BITS) | this.seq;
  }
}

const EPOCH_MS = 1704067200000; // must match the generator's epoch

export function decode(id: bigint) {
  const seq = id & 0xfffn;               // low 12 bits
  const workerId = (id >> 12n) & 0x3ffn; // next 10 bits
  const tsOffset = id >> 22n;            // high 41 bits
  const createdAtMs = Number(tsOffset) + EPOCH_MS;
  return {
    createdAt: new Date(createdAtMs),
    createdAtMs,
    workerId: Number(workerId),
    sequence: Number(seq),
  };
}

CREATE TABLE worker_leases (
  worker_id INT PRIMARY KEY CHECK (worker_id BETWEEN 0 AND 1023),
  hostname TEXT NOT NULL,
  leased_until TIMESTAMPTZ NOT NULL
);

-- Claim the first free or expired worker id atomically
INSERT INTO worker_leases (worker_id, hostname, leased_until)
SELECT gs.id, 'api-7', now() + interval '60 seconds'
FROM generate_series(0, 1023) AS gs(id)
WHERE NOT EXISTS (
  SELECT 1 FROM worker_leases w
  WHERE w.worker_id = gs.id AND w.leased_until > now()
)
ORDER BY gs.id
LIMIT 1
ON CONFLICT (worker_id) DO UPDATE
  SET hostname = EXCLUDED.hostname, leased_until = EXCLUDED.leased_until
  WHERE worker_leases.leased_until <= now()
RETURNING worker_id;