Design a URL Shortener (bit.ly)

1Requirements

2Back-of-envelope estimation

3API design

4High-level design

Clients hit a load balancer that fronts a fleet of stateless API servers. Because the servers hold no session state, we can scale them horizontally behind the balancer and any server can handle any request. Writes (create URL) and reads (redirect) can be served by the same fleet, or split into separate services so the huge read volume never starves writes.

On the write path, the API server obtains a unique ID and encodes it in Base62 to produce the short code. The cleanest approach is a Key Generation Service: an offline worker pre-generates batches of unique codes and stores them in a key database; API servers grab a batch of unused keys into memory and hand them out with zero collision risk and no coordination per request. The mapping (short_code → long_url, owner, expiry) is written to the primary datastore.

On the read path, the server first checks a distributed cache (Redis) keyed by short code. On a hit, it issues the redirect immediately. On a miss, it reads the datastore, populates the cache with a TTL, and redirects. With a 100:1 read ratio and a heavily skewed popularity distribution, cache hit rates above 90% are realistic, which keeps p99 latency low and shields the database.

The datastore itself can be a simple key-value store (DynamoDB, Cassandra) or sharded MySQL, since the access pattern is a single-key lookup with no joins. Shard by hash of the short code for even distribution. Click events are not written synchronously on the redirect path; instead the server emits an event to a message queue, and an analytics consumer aggregates counts in batches, keeping redirects fast.

5Data model

6Deep dives

7Rapid implementation: build the MVP

Theory is table stakes. Here is how you would stand up a working version fast, on a budget, with the core algorithm in real code.

1. 01Create a urls table: id BIGSERIAL PK, short_code VARCHAR(7) with a UNIQUE index, long_url TEXT, created_at, expires_at NULL.
2. 02Write a base62 encoder that turns the auto-increment id into a short code, offset by a large constant (e.g. 100000000) so codes start at 5-6 chars.
3. 03Build POST /api/urls: validate the URL with the URL constructor, insert the row, encode the returned id, update the row with the code, return short URL.
4. 04Support custom aliases by inserting the alias directly and catching the Postgres 23505 unique-violation error to return 409.
5. 05Build GET /[code]: look up Redis first, fall back to Postgres, SET the code in Redis with a 24h TTL, respond with a 302 redirect.
6. 06Fire-and-forget an INCR on clicks:{code} in Redis on each redirect; flush counts to Postgres with a cron every minute.
7. 07Add a nightly cron that deletes or deactivates rows where expires_at < now().
8. 08Point Cloudflare at the app and enable caching of 404s to blunt enumeration scans.

const ALPHABET = "0123456789abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ";
const OFFSET = 100_000_000n; // avoid tiny 1-2 char codes

export function encodeBase62(id: bigint): string {
  let n = id + OFFSET;
  let out = "";
  while (n > 0n) {
    out = ALPHABET[Number(n % 62n)] + out;
    n = n / 62n;
  }
  return out;
}

export function decodeBase62(code: string): bigint {
  let n = 0n;
  for (const ch of code) {
    n = n * 62n + BigInt(ALPHABET.indexOf(ch));
  }
  return n - OFFSET;
}

async function createShortUrl(longUrl: string, alias?: string) {
  new URL(longUrl); // throws on invalid input
  if (alias) {
    try {
      await sql(
        "INSERT INTO urls (short_code, long_url) VALUES ($1, $2)",
        [alias, longUrl]
      );
      return alias;
    } catch (e: any) {
      if (e.code === "23505") throw new Error("alias taken"); // 409
      throw e;
    }
  }
  // id-based codes cannot collide: insert first, derive code from id
  const rows = await sql(
    "INSERT INTO urls (short_code, long_url) VALUES ('pending', $1) RETURNING id",
    [longUrl]
  );
  const code = encodeBase62(BigInt(rows[0].id));
  await sql("UPDATE urls SET short_code = $1 WHERE id = $2", [code, rows[0].id]);
  return code;
}

export async function GET(req: Request, ctx: { params: { code: string } }) {
  const { code } = ctx.params;
  let longUrl = await redis.get("url:" + code);
  if (!longUrl) {
    const rows = await sql(
      "SELECT long_url FROM urls WHERE short_code = $1 AND (expires_at IS NULL OR expires_at > now())",
      [code]
    );
    if (rows.length === 0) return new Response("Not found", { status: 404 });
    longUrl = rows[0].long_url;
    await redis.set("url:" + code, longUrl, { ex: 86400 });
  }
  redis.incr("clicks:" + code); // not awaited, off the hot path
  return Response.redirect(longUrl, 302);
}