23. Why can a model with a higher eval score still be blocked?
- Other guardrails, such as refusal rate or cost, got worse
- Higher scores are always fake
- New models cannot be used
- Eval scores do not matter
Show answer
Answer: Other guardrails, such as refusal rate or cost, got worse
Check every guardrail, not only quality.
Lesson: Model Upgrades and Prompt Changes
25. Which item belongs on an AI rollout checklist?
- Skipping slice-level evaluation
- Launching to 100% on day one
- A tested kill switch with a designed fallback experience
- Hiding that content is AI-generated
Show answer
Answer: A tested kill switch with a designed fallback experience
Control, measure, communicate.
Lesson: An AI Rollout Checklist