Lesson 14 / 25

Collector Configuration

Defining and enabling pipelines.

Define components, then wire them in service

A Collector configuration file declares components under receivers, processors, exporters, connectors and extensions, then enables them in service.pipelines. Components defined but not used in a pipeline are ignored. The recommended processor order starts with memory_limiter and ends with batch. The debug exporter prints data to the Collector's logs while testing, and the validate command checks a configuration file.

A complete Collector configuration

Receives OTLP, sends traces to Tempo and metrics to a Prometheus-compatible backend.

receivers:
  otlp:
    protocols:
      grpc:
        endpoint: 0.0.0.0:4317
      http:
        endpoint: 0.0.0.0:4318

processors:
  memory_limiter:
    check_interval: 1s
    limit_percentage: 80
    spike_limit_percentage: 20
  batch: {}

exporters:
  otlp/tempo:
    endpoint: tempo:4317
    tls:
      insecure: true            # demo only; use TLS in production
  prometheusremotewrite:
    endpoint: http://mimir:9009/api/v1/push
  debug:
    verbosity: basic

service:
  pipelines:
    traces:
      receivers: [otlp]
      processors: [memory_limiter, batch]
      exporters: [otlp/tempo]
    metrics:
      receivers: [otlp]
      processors: [memory_limiter, batch]
      exporters: [prometheusremotewrite]
    logs:
      receivers: [otlp]
      processors: [memory_limiter, batch]
      exporters: [debug]

Name repeated components with a slash

otlp/tempo and otlp/vendor are two independent OTLP exporters with different settings.

Quick check: What happens to a processor defined in the configuration but not listed in any pipeline?

  • It becomes a receiver
  • It applies to all pipelines
  • The Collector deletes it
  • It is not used
Answer

It is not used — Pipelines enable components.