Lesson 12 / 25

Oversight and Human-in-the-Loop Costs

People keep AI accurate and safe.

Review, monitoring, escalations

Automation needs people: reviewers checking flagged items (already counted in the task-level time if you decomposed properly), staff handling exceptions and escalations, and someone monitoring quality, drift and incidents. Count these explicitly. If review time is counted in the benefit calculation (as minutes remaining per item), do not count it again as a cost line; state where each appears.

Autopilot still needs pilots

Planes fly on autopilot most of the time, but airlines still budget for trained pilots who monitor and take over.

State where review time is counted

Write "review time included in minutes per item" or "review as separate cost line" so reviewers can check there is no double counting.

Quick check: Why must oversight be included in AI automation costs?

  • AI never makes mistakes
  • People are still needed to review, handle exceptions and monitor quality
  • Oversight is free
  • It is only needed for software bugs
Answer

People are still needed to review, handle exceptions and monitor quality — Humans in the loop cost money.