# Success Metrics and Guardrail Metrics — Safe Rollout Plans for AI Features

Source: https://www.skillbyai.com/en/ai-feature-rollouts/w-metrics

> What should improve, and what must not get worse.

## Two kinds of numbers

**Success metrics** capture the value the feature should create: task completion, time saved, conversion, satisfaction. **Guardrail metrics** protect against harm: error rate, thumbs-down rate, escalations, safety flags, latency, cost per request, complaint volume. A launch should improve success metrics without breaching any guardrail. Define each metric precisely (numerator, denominator, time window), with a threshold and a source, and make sure you can measure it before day one.

## Speed and seatbelts

A faster car is the goal; seatbelts, brakes and crash tests make sure the speed does not come at an unacceptable cost.

## Instrument before launch

If a metric cannot be measured on the first day of dogfooding, it cannot protect the launch.

**Quiz:** Which is a guardrail metric for an AI feature?

- [ ] Office temperature
- [ ] Number of marketing emails sent
- [ ] Lines of code written
- [x] Rate of safety flags per thousand responses

*Answer:* Rate of safety flags per thousand responses. Guardrails measure harm and risk.