Generating Unique IDs

Snowflake-style IDs

A single auto-increment counter becomes a bottleneck and a single point of failure across shards. Options: UUIDs (random, 128 bits, no coordination, but not time-ordered, which hurts index locality; UUIDv7 is time-ordered), ID ranges handed to each server, or Snowflake-style 64-bit IDs combining a millisecond timestamp, a machine ID and a per-millisecond sequence. These are roughly time-sortable and generated locally, but depend on reasonably synchronised clocks.

Building and parsing Snowflake-style IDs, run

I ran this with Python 3.12.3 using only the standard library; inputs are fixed or seeded, so the output is reproducible. IDs created later compare as larger; a machine can create 4,096 IDs per millisecond, and 41 timestamp bits last about 70 years from the custom epoch.

# Snowflake-style 64-bit IDs: 41-bit ms timestamp | 10-bit machine | 12-bit sequence
EPOCH_MS = 1_704_067_200_000          # 2024-01-01T00:00:00Z, a custom epoch

def make_id(ts_ms, machine, seq):
    return ((ts_ms - EPOCH_MS) << 22) | (machine << 12) | seq

def parse(i):
    return {"ts_ms": (i >> 22) + EPOCH_MS, "machine": (i >> 12) & 0x3FF, "seq": i & 0xFFF}

ts = 1_790_000_000_000                # a fixed timestamp for the demo
a, b, c = make_id(ts, 7, 0), make_id(ts, 7, 1), make_id(ts + 1, 3, 0)
print(a, b, c)
print(a < b < c, "-> sortable by time")
print(parse(b))
print(f"ids per machine per ms: {2 ** 12:,}")
print(f"timestamp range: {2 ** 41 / (1000 * 3600 * 24 * 365.25):.1f} years")

Output:

360428286771228672 360428286771228673 360428286775406592
True -> sortable by time
{'ts_ms': 1790000000000, 'machine': 7, 'seq': 1}
ids per machine per ms: 4,096
timestamp range: 69.7 years

Mention clock skew

If a clock moves backwards, a generator must wait or refuse to issue IDs to avoid duplicates.

Quick check: What advantage do Snowflake IDs have over random UUIDv4s?

• They never need a machine ID
• They are roughly sortable by creation time
• They are 256 bits long
• They require a central counter