# Alerting Rules — Prometheus + Grafana

Source: https://www.skillbyai.com/en/prometheus-grafana/al-rules

> Conditions with a duration.

## expr, for, labels and annotations

An alerting rule has a PromQL `expr` that returns series when something is wrong, a `for` duration the condition must hold (to avoid alerts on brief blips; the alert is **pending** until then and **firing** after), `labels` such as severity used for routing, and `annotations` with a human-readable summary, description and runbook link. Templates like `{{ $labels.instance }}` and `{{ $value }}` fill in details.

## Wake people only when it matters

Alerting rules detect problems, Alertmanager routes them, and good alert design prevents fatigue.

![Three ideas: alerting rules, Alertmanager, designing good alerts.](assets/figures/prometheus-grafana/section-6-map.svg) — Figure 6.1 — Alert rules, routing and alert design.

## Two alerting rules

Error ratio and a down target.

```yaml
groups:
  - name: orders-api-alerts
    rules:
      - alert: OrdersApiHighErrorRatio
        expr: job:http_requests_error_ratio:rate5m{job="orders-api"} > 0.05
        for: 10m
        labels:
          severity: page
        annotations:
          summary: "Orders API error ratio above 5%"
          description: "Error ratio is {{ $value | humanizePercentage }} for 10 minutes."
          runbook_url: "https://runbooks.example.com/orders-api/high-errors"
      - alert: TargetDown
        expr: up == 0
        for: 5m
        labels:
          severity: ticket
        annotations:
          summary: "{{ $labels.job }} target {{ $labels.instance }} is down"
```

## Every paging alert needs a runbook

A link explaining what to check and how to mitigate shortens incidents, especially at night.

**Quiz:** What does the for clause do in an alerting rule?

- [ ] Sets the scrape interval
- [ ] Repeats the notification
- [x] Requires the condition to stay true for that long before firing
- [ ] Deletes the alert after that time

*Answer:* Requires the condition to stay true for that long before firing. Filters brief blips.