# How Prometheus Works — Prometheus + Grafana

Source: https://www.skillbyai.com/en/prometheus-grafana/f-arch

> Pull-based scraping and a local TSDB.

## Scrape, store, query, alert

The Prometheus server periodically **scrapes** (HTTP GET) a `/metrics` endpoint on each target, typically every 15 to 60 seconds, and stores the samples in its local **time-series database** (TSDB). Targets come from static configuration or **service discovery** (Kubernetes, cloud APIs, Consul). Prometheus evaluates **recording and alerting rules**, sends firing alerts to **Alertmanager**, and answers **PromQL** queries from its web UI, its HTTP API and Grafana. Short-lived batch jobs can push to a Pushgateway, which Prometheus then scrapes.

## The main components

How data flows.

```text
apps / exporters  --(/metrics, pulled every scrape_interval)-->  Prometheus server
                                                                   |  TSDB (local disk)
                                                                   |  rules (recording, alerting)
                                         Alertmanager  <--alerts---|
                                     (email, Slack, PagerDuty)     |
                                                    Grafana  --PromQL queries-->  Prometheus
```

## Pull makes health visible

Because Prometheus scrapes targets, a failed scrape is itself a signal: the up metric becomes 0.

**Quiz:** How does Prometheus normally collect metrics?

- [x] It pulls them by scraping HTTP endpoints
- [ ] Applications push every sample to it
- [ ] It reads log files
- [ ] It queries databases directly

*Answer:* It pulls them by scraping HTTP endpoints. Pull model with service discovery.