LLM Observability (Langfuse / LangSmith) Interview Questions Hindi mein

जवाब: A wrong or unhelpful answer usually still arrives as a successful response

Hallucinations, refusals and off-topic answers come back as normal successful responses, so you need to look at content and quality, not only status codes.

जवाब: Input and output token counts for a model call

Duration, status and service name are standard APM fields. Token usage is LLM-specific and drives both cost and context-window limits.

जवाब: The prompt version and model configuration on each trace

Configuration signals (prompt version, model, parameters) let you group traces before and after a change and pinpoint the cause.

जवाब: They are siblings: both are children of the root span

Both were opened while the root was the current span, so both record the root as their parent, making them siblings.

जवाब: Token usage and cost analytics aggregate over model-call records with model and usage fields

Cost and token dashboards rely on the model and usage fields that the generation type carries. Plain spans still support input and output, but lack those semantics.

जवाब: A stable internal user identifier

A stable internal ID lets you group a user's traces without sending personal data to the observability backend. A random per-request value defeats grouping.

जवाब: Trace data containing sensitive information must stay inside its own infrastructure

Data residency and privacy are the usual reasons. The SDK, token tracking and API keys work the same way with both options.

जवाब: One trace with `answer` as the parent and `retrieve` nested inside it

The SDK tracks the current observation in context, so the inner decorated call becomes a child of the outer one, and the outermost call defines the trace.

जवाब: The process exits before the background queue is flushed

The SDK batches events in the background; a short-lived process must call `flush()` before it exits or the final batches are lost.

जवाब: It can trace plain Python or JS/TS code through its SDK, not only LangChain apps

The `langsmith` SDK's `@traceable` decorator and client wrappers work with any code; LangChain integration is a convenience, not a requirement.

जवाब: retriever

The `retriever` run type tells LangSmith the output is a list of documents, so it renders their content and metadata.

जवाब: Wrap the helper with `@traceable` so it joins the current trace

Auto tracing only covers framework components. Decorating your own function makes it a child run of the currently active trace.

जवाब: Generation: the prompt, context ordering or model choice

Retrieval did its job because the right chunk was present. The failure happened when the model used the context, so look at the prompt and generation step.

जवाब: The agent is looping without making progress; check tool results and prompt instructions

Repeated near-identical tool calls are a classic agent loop. Look at what the tool returned and whether the model is told how to proceed when results are empty or unhelpful.

जवाब: The same instrumentation can export to different observability backends

OTel decouples instrumentation from the backend. The GenAI conventions are still evolving, so names can change, and they do not affect model speed.

जवाब: 0.004 units

2,000 x 1 / 1,000,000 = 0.002, plus 500 x 4 / 1,000,000 = 0.002, giving 0.004 units.

जवाब: It shows the slow tail that a meaningful share of users actually experience

A few slow outliers distort the mean, while p95 directly states that 5% of requests are slower than this value, which is what users complain about.

जवाब: Alert when average user-feedback score drops sharply compared with the previous week

Trend-based alerts on user-visible symptoms catch real regressions. Single thumbs-downs are normal noise and belong in a review queue, not a pager.

जवाब: The ID of the trace or run that produced the answer

Feedback is useful only when linked to the exact trace or run, which is why apps pass the trace or run ID to the frontend.

जवाब: A specific rubric with a small discrete scale, validated against human labels

Clear rubrics, discrete scales and agreement checks with humans reduce noise. A faithfulness judge needs the context to check claims against it.

जवाब: To catch regressions by comparing scores with the previous version on the same examples

Running both versions on the same examples gives a fair comparison and shows which cases improved or broke.

जवाब: It reveals how the system handles real, previously unseen user inputs

Online evaluation scores real traffic, including new question types and drift. Controlled comparisons are the strength of offline experiments.

जवाब: Makes apps that fetch the production label start using version 7, without a code change

Labels are pointers to versions. Apps fetching by label pick up the new version (after their cache refreshes) while old versions remain available for rollback.

जवाब: So a single conversation does not switch prompts mid-way, keeping behaviour and comparisons consistent

Switching variants within a conversation confuses users and mixes two versions inside one session's traces, making the comparison unreliable.

जवाब: In the SDK's masking function, before data leaves the application

Masking in the SDK stops the raw values from ever being sent or stored. The other options leave the data stored somewhere first.

जवाब: Per-span sampling produces incomplete trees where some steps of a request are missing

If each span is sampled independently, kept traces lose random steps, which makes them misleading for debugging. Decide once per trace and apply it to every span.

जवाब: Trace context was not propagated to the worker

Spans attach to whatever trace context is current. In a separate worker or service that context is absent unless you pass and re-attach it.

जवाब: Whether the tool can be self-hosted within its own infrastructure

Data residency is a hard requirement here, so a self-hostable option (or a contract that guarantees it) comes first. The other factors are preferences.

जवाब: Trace: one end-to-end request made of nested spans

A trace is one request's tree of spans. A session groups many traces, a generation is a model-call span, and a dataset is a fixed set of examples.

जवाब: Use traces to attribute cost by feature, step and token type to find where money actually goes

Measure before optimising. Traces show whether cost comes from long prompts, history, retries or agent loops, and that decides the right fix.

जवाब: Ensure tracing keys are set in production and call flush() before the handler returns

Silent loss in production usually means missing configuration or unflushed background batches when the runtime freezes the process.