LLM observability should explain why a system failed

Traditional service metrics are necessary but incomplete for an LLM application. A request can return HTTP 200 while the model cites irrelevant evidence, misunderstands intent, chooses the wrong tool or produces an answer that is fluent but unsupported.

Observability therefore has to capture semantic behavior as well as infrastructure behavior.

A useful trace records user intent, prompt version, model choice, retrieved context, tool calls, structured-output validation, retries, latency, token usage and the final answer. For RAG systems, the retrieved evidence and its ranking are especially important.

When this context is attached to each request, failures become reproducible. Teams can compare successful and unsuccessful runs instead of relying on screenshots or anecdotal reports.

Online monitoring should catch operational regressions such as rising latency, higher cost, tool failures, empty retrievals and changes in user behavior. Offline evaluation should measure quality on a curated dataset before a model, prompt or retrieval change reaches production.

Useful evaluation dimensions include groundedness, answer relevance, retrieval quality, structured-output validity and task completion. The exact score matters less than having a stable benchmark that makes regressions visible.

User feedback is most valuable when it can be connected to the exact prompt, retrieval result and tool path that produced the answer. A thumbs-down without execution context tells a team that something went wrong, but not why.

Closing that loop turns production feedback into engineering evidence and helps prioritize the changes that actually improve user outcomes.