Dashboards as Code

Treat dashboards like software

Hand-edited dashboards drift and break. Grafana provisioning loads data sources and dashboards from files at start-up, so they can be version-controlled and reviewed. Export dashboards as JSON into a repository, or generate them with tools such as Grafonnet (Jsonnet) or the Grafana Terraform provider. On Kubernetes, sidecars commonly load dashboards from ConfigMaps. Community dashboards (for example for node_exporter) are useful starting points.

Provisioning files

A data source and a dashboard folder.

# /etc/grafana/provisioning/datasources/prometheus.yml
apiVersion: 1
datasources:
  - name: Prometheus
    type: prometheus
    access: proxy
    url: http://prometheus:9090
    isDefault: true

# /etc/grafana/provisioning/dashboards/default.yml
apiVersion: 1
providers:
  - name: default
    folder: Services
    type: file
    options:
      path: /var/lib/grafana/dashboards     # JSON dashboards from the repository

Review dashboard changes in pull requests

Diffs of dashboard JSON or Jsonnet catch broken queries before they reach the on-call team.

Quick check: What does Grafana provisioning do?

• Scrapes metrics
• Loads data sources and dashboards from files at start-up
• Sends alerts to PagerDuty
• Compresses the TSDB