# User Feedback and Scores — LLM Observability (Langfuse / LangSmith)

Source: https://www.skillbyai.com/en/llm-observability/lo-feedback

> Collect explicit and implicit user feedback and attach it to traces as scores.

## Turning reactions into data

The cheapest quality signal is your users. **Explicit feedback** is what they tell you: thumbs up or down, a star rating, a correction they type. **Implicit feedback** is what they do: copying the answer, retrying the same question, abandoning the chat, or escalating to a human agent. To be useful, feedback must be linked to the **exact trace** that produced the answer. The usual pattern: your backend returns the trace ID (or a run ID you generated in advance) with the response; the frontend sends it back with the user's rating; your backend writes a **score** (Langfuse) or **feedback** (LangSmith) against that ID. Keep in mind the biases: most users never rate, and angry users rate more often than happy ones, so feedback is better for finding bad examples and comparing versions than for an absolute quality number. Combine it with automated evaluation for coverage.

## Feedback in LangSmith with a pre-set run ID

Generating the run ID up front lets you hand it to the frontend before the user reacts. `answer` is a `@traceable` function.

```python
import uuid
from langsmith import Client

client = Client()
run_id = uuid.uuid4()

reply = answer("Where is my order?", langsmith_extra={"run_id": run_id})
# ... send reply and str(run_id) to the frontend ...

# later, when the user clicks thumbs-up:
client.create_feedback(run_id, key="user-thumbs", score=1, comment="helpful")
```

## Comparing feedback by version

Aggregating raw feedback events per prompt version. Note how small these samples are.

```python
from collections import defaultdict

feedback = [("v3", 1), ("v3", 0), ("v4", 1), ("v4", 1), ("v3", 1), ("v4", 0), ("v4", 1)]
stats = defaultdict(lambda: [0, 0])          # version -> [thumbs_up, total]
for version, thumbs_up in feedback:
    stats[version][0] += thumbs_up
    stats[version][1] += 1
for version, (up, n) in sorted(stats.items()):
    print(f"{version}: {up}/{n} positive ({up / n:.0%})")
```

Output:

```
v3: 2/3 positive (67%)
v4: 3/4 positive (75%)
```

## Small samples lie

75% versus 67% from seven ratings means nothing; one extra click flips it. Wait for enough feedback, look at confidence intervals or use a proper experiment, and read the negative traces before declaring a winner.

**Quiz:** What must a feedback event include so it can be attached to the right answer?

- [x] The ID of the trace or run that produced the answer
- [ ] The user's full chat history
- [ ] The model's API key
- [ ] The time the frontend was deployed

*Answer:* The ID of the trace or run that produced the answer. Feedback is useful only when linked to the exact trace or run, which is why apps pass the trace or run ID to the frontend.