पाठ 13 / 35

How Prometheus Works

यह पाठ अभी हिंदी में अनुवादित नहीं है, इसलिए अंग्रेज़ी में दिखाया जा रहा है।

Explain Prometheus's pull model, service discovery, the up metric and where its local storage fits.

लेखक: · 15 साल से वेब डेवलपर

Pull, store, evaluate

Prometheus is an open-source monitoring system and time-series database, and a graduated CNCF project. Its core loop is simple. Using service discovery (static lists, Kubernetes, cloud APIs, DNS) it finds targets. Every scrape_interval it sends an HTTP GET to each target's /metrics endpoint, parses the text, attaches labels such as job and instance, and appends the samples to its local TSDB on disk.

Pulling has useful side effects: Prometheus knows when a target is unreachable, and records this in an automatic metric up (1 for a successful scrape, 0 for a failed one). Short-lived batch jobs that finish before a scrape can push to a Pushgateway instead, but that is the exception.

Prometheus also evaluates recording rules (precompute expensive queries) and alerting rules on a schedule, sending firing alerts to Alertmanager. A single Prometheus server is not designed for long-term, highly available storage; projects such as Thanos, Cortex, Grafana Mimir and VictoriaMetrics add that, typically fed by remote_write.

One Prometheus evaluation cycle

Service discovery produces a target list, Prometheus scrapes each /metrics endpoint, samples land in the TSDB, rules are evaluated against stored data, and firing alerts go to Alertmanager.

Five stages in order: service discovery finds targets, Prometheus scrapes them, samples are stored in the TSDB, rules are evaluated, alerts are sent to Alertmanager.
Figure 5.1: service discovery -> scrape -> TSDB -> rules -> Alertmanager

A prometheus.yml

Two jobs: one with a static target list and an extra env label, one that discovers every pod through the Kubernetes API. In practice the pod job also needs relabel_configs to choose which pods and ports to scrape.

global:
  scrape_interval: 15s       # how often to pull every target
  evaluation_interval: 15s   # how often to evaluate rules

rule_files:
  - "rules/*.yml"

scrape_configs:
  - job_name: "checkout"
    metrics_path: /metrics
    static_configs:
      - targets: ["checkout-1:8000", "checkout-2:8000"]
        labels:
          env: "prod"

  - job_name: "kubernetes-pods"
    kubernetes_sd_configs:
      - role: pod

Alert on up == 0 first

If a target cannot be scraped, all its other metrics simply stop, and rules like "error rate > 5%" quietly evaluate to nothing. A basic up == 0 alert (and absent() for critical metrics) protects you from the silent failure where monitoring itself is blind.

त्वरित जाँच: What does the automatically generated `up` metric tell you?

  • Whether the last scrape of a target succeeded
  • The uptime of the host in seconds
  • How many users are currently online
  • The number of alerts currently firing
Answer

Whether the last scrape of a target succeeded — Prometheus sets up to 1 when a scrape succeeds and 0 when it fails, which makes it the simplest way to detect a dead or unreachable target.