Performance Overhead and Sampling

Tracing should never break the request

Good LLM observability SDKs do not block your request while sending data: they put events on an in-memory queue and a background thread batches and uploads them. The extra latency per request is therefore small, and if the backend is down, the SDK should drop or retry data rather than fail your user's call. Volume still costs money: storage, network and per-event pricing on managed plans. Sampling keeps only a fraction of traces. Langfuse's SDK supports a sample rate setting, and you can also sample yourself. Sampling must be decided per trace, not per span, otherwise you keep half-trees. Use a deterministic decision based on the trace ID so every service agrees. Trade-offs: sampled-out traces cannot be debugged, and rare failures may be missed, so many teams keep 100% for errors, negative feedback and low-volume features, and sample only high-volume, healthy traffic.

Deterministic sampling by trace ID

Hashing the trace ID gives every service the same keep-or-drop answer, so traces are never half-sampled.

import hashlib

def keep_trace(trace_id: str, rate: float) -> bool:
    bucket = int(hashlib.sha256(trace_id.encode()).hexdigest()[:8], 16)
    return bucket / 0xFFFFFFFF < rate

ids = [f"trace-{i}" for i in range(10_000)]
print("kept", sum(keep_trace(t, 0.1) for t in ids), "of", len(ids))
print("stable:", keep_trace("trace-42", 0.1) == keep_trace("trace-42", 0.1))

Output:

kept 988 of 10000
stable: True

Ticket checker on a busy train

A TTE cannot check every passenger on a packed train, so they check some coaches fully. Checking half the people in every coach would tell you less. Sample whole traces, like whole coaches.

Quick check: Why should the sampling decision be made per trace rather than per span?

• Spans cannot be sampled at all
• Per-span sampling increases token cost
• Per-trace sampling is required by the HTTP specification
• Per-span sampling produces incomplete trees where some steps of a request are missing