# RED, USE and the Four Golden Signals — Logging, Metrics, Tracing & Alerting

Source: https://www.skillbyai.com/en/observability/ob-red-use-golden

> Apply the RED, USE and golden-signals checklists to decide which metrics a service or resource must expose.

## Checklists instead of guesswork

Teams new to metrics either measure nothing or measure everything. Three well-known checklists help.

**RED** (for request-driven services): **Rate** (requests per second), **Errors** (failed requests per second or as a ratio), **Duration** (latency distribution, usually as a histogram). If every service exposes RED, any dashboard can compare them.

**USE** (for resources such as CPU, disks, network links, connection pools): **Utilisation** (how busy, as a percentage of time or capacity), **Saturation** (how much work is waiting: run-queue length, pool wait time), **Errors** (device or resource errors). Saturation is often the earliest sign of trouble: a pool at 100% utilisation with a growing queue means requests are now waiting.

The **four golden signals** from Google's SRE book combine both: **latency, traffic, errors, saturation**. One subtle point it highlights: measure the latency of failed requests separately, because a fast error can make average latency look better while users suffer.

## A railway ticket counter

For the counter as a *service*: how many people are served per minute (rate), how many leave without a ticket (errors), how long each takes (duration). For the clerk as a *resource*: what fraction of time they are busy (utilisation), how long the queue is (saturation), how often the printer jams (errors). A clerk can be 100% busy and still fine; a queue stretching out of the station is the real alarm.

## RED in PromQL

Three queries give a RED view of one service, using the counter and histogram from earlier in this course.

```promql
# Rate: requests per second
sum(rate(http_requests_total{job="checkout"}[5m]))

# Errors: fraction of requests returning 5xx
sum(rate(http_requests_total{job="checkout", status=~"5.."}[5m]))
  / sum(rate(http_requests_total{job="checkout"}[5m]))

# Duration: 95th percentile latency in seconds
histogram_quantile(0.95,
  sum by (le) (rate(http_request_duration_seconds_bucket{job="checkout"}[5m])))
```

**Quiz:** In the USE method, which measurement describes saturation of a database connection pool?

- [ ] The number of queries that returned an error
- [x] The number of requests waiting to obtain a connection
- [ ] The total queries executed since startup
- [ ] The size of the database on disk

*Answer:* The number of requests waiting to obtain a connection. Saturation is queued or waiting work. Requests waiting for a free connection show the pool cannot keep up, even before errors appear.
