Lesson 25 / 35
Dashboards That Answer Questions
Design dashboards with a clear purpose, a top-down layout and honest visualisations instead of walls of graphs.
By Rahul Agarwal · 15 years in web development
Every dashboard needs a question
A dashboard is a set of saved queries arranged for a purpose. Tools like Grafana make adding panels so easy that teams end up with fifty graphs nobody can read during an incident.
Start from the reader's question. A service overview answers "is this service healthy?": the SLO and remaining error budget at the top, then RED panels (request rate, error ratio, latency percentiles), then saturation of key dependencies. A debugging dashboard for the same service breaks these down by route, pod, region and dependency, with template variables so one dashboard serves every environment. Link the overview to the drill-down, and link panels to logs and traces.
Design details matter. Put the most important panel top-left. Use consistent units (seconds, not a mix of ms and s) and show thresholds as lines. Graph percentiles, not just averages. Use the same time range across panels so spikes line up. Avoid pie charts for time-varying data and avoid stacking lines that should be compared. Keep dashboards in version control as JSON or generated code, so they are reviewed like any other change.
Dashboards are not alerts
Nobody watches a screen at 3 a.m. If a condition matters, write an alert for it; dashboards are for investigating after you are notified and for capacity reviews. A good test: during the last incident, which panels did people actually open? Remove or move the rest.
A car's instrument cluster
A car shows speed and fuel prominently, a few warning lights, and hides everything else behind a diagnostic port. Mechanics plug in a scanner for detail. A service overview dashboard is the instrument cluster; the debugging dashboard is the mechanic's scanner.
Quick check: What belongs at the top of a service overview dashboard?
- A pie chart of HTTP methods
- Per-pod garbage collection pauses
- SLO status and the RED signals for the service
- The raw log stream
Answer
SLO status and the RED signals for the service — An overview should answer "are users OK?" at a glance: SLO and error budget, request rate, errors and latency. Detailed breakdowns belong on drill-down dashboards.