Lesson 25 / 30

Common Estimation Mistakes

Recognise the errors that most often break estimates and correct them quickly.

By · 15 years in web development

The usual suspects

Most bad estimates fail in a handful of predictable ways. Unit slips: bits versus bytes, per day versus per second, KB versus MB; these produce errors of 8x or 1,000x. Planning for averages: sizing to average QPS instead of peak, or to average object size when a long tail of large objects dominates storage. Forgetting multipliers: replication, indexes, backups, media variants, retries and internal fan-out each add a factor, and together they compound. Ignoring growth: sizing for launch day and running out in three months. Treating one resource as the only resource: computing CPU but not connections, memory, IOPS or egress. False precision: quoting 3,472.2 servers suggests accuracy you do not have; round and give a range. The remedy is mechanical: carry units on every line, list multipliers explicitly, and sanity-check each result per user and per machine.

A four-point checklist

Before trusting an estimate, run it through a unit check, a peak check, a multiplier check and a per-user check.

Four connected stages: unit check, peak check, multiplier check, per-user check.
Figure 9.1: unit check -> peak check -> multiplier check -> per-user check

Catching a units bug

A storage estimate written in two ways: one with an MB/KB slip, one with units carried explicitly.

uploads_per_day = 1_000_000
photo_size_kb = 300

wrong_gb = uploads_per_day * photo_size_kb / 1e9      # treats KB as bytes
right_gb = uploads_per_day * photo_size_kb * 1e3 / 1e9
print(f"wrong: {wrong_gb:.1f} GB/day")
print(f"right: {right_gb:.0f} GB/day")
print(f"per user check: {right_gb * 1e9 / uploads_per_day / 1e3:.0f} KB per upload")

Output:

wrong: 0.3 GB/day
right: 300 GB/day
per user check: 300 KB per upload

Give a range, not a point

Say 'roughly 300 to 600 TB over three years' rather than '412.7 TB'. A range communicates uncertainty honestly and shows you understand which assumptions drive the answer.

Quick check: An estimate says each user will store 40 GB of text messages per day. What is the most likely explanation?

  • Text messages are larger than expected
  • A unit conversion error, such as multiplying by 1,000 one time too many
  • The replication factor was set to 3
  • The retention period is too short
Answer

A unit conversion error, such as multiplying by 1,000 one time too many — A per-user sanity check exposes the absurdity: nobody types 40 GB of text a day. That magnitude of error almost always comes from a unit slip.