Ask a team shipping an agent what happens when it misbehaves and you will usually get a good answer: open the trace, read the spans, find the tool call that went sideways. Ask the same team how many of yesterday's ten thousand runs went sideways and the answer is quieter. LangChain's State of Agent Engineering survey found 89% of agent teams had observability in place and 52% had evaluations. That 37-point gap is where most launch-day surprises live.
What observability actually tells you
A trace is a record of what happened: the prompt, the model's response, each tool call with its arguments and result, timings, token counts. It is indispensable for debugging a single run. It is also, on its own, silent on the only question leadership cares about, which is whether the run was correct.
Correctness is a judgment. Somebody has to decide that calling search_orders before get_customer was wrong, that the refund amount was ten times too large, that stopping after two steps was premature. Tracing tools record; they do not judge. The vendors know this, which is why every one of them has bolted on an "evals" tab. But the tab is a place to put graders, not the graders themselves.
What an eval adds
An eval is three things: a case (input plus context), an expectation (what a correct trajectory or outcome looks like), and a grader (code or a calibrated model that scores the actual against the expected). Run a few hundred of them on every change and you have a regression suite. Run none and every prompt tweak is a coin flip you find out about from a customer.
For agents specifically, "expectation" has more than one layer:
- Outcome. Did the user get the right answer or the right side effect?
- Trajectory. Did the agent call the right tools, with the right arguments, in a sensible order, and stop when it should?
- Side effects. What changed in connected systems, and was every change intended?
Teams coming from chatbot evaluation tend to grade only the first layer. Agents fail in the second and third far more often. An agent that produces the right summary after deleting the wrong record is not a pass.
Why the gap persists
Three reasons come up on nearly every engagement.
Nobody owns the definition of correct. Product knows what the agent should do; engineering knows what it did. Writing the expectation down requires both in one room for a few hours, and that meeting rarely gets scheduled.
Deterministic grading feels impossible. Outputs vary, so teams assume every grader has to be an LLM. In practice most agent failures are visible in structured data: tool names, argument values, call counts, state after the run. Those can be checked exactly. Save the model-graded judgments for the genuinely fuzzy cases.
The dataset does not exist. Real cases with real expectations take effort to curate. The shortcut is to mine your own traces: sample a few hundred runs, label them, and you have both a failure taxonomy and the seed of a dataset.
Closing the gap
Here is the order that works. It is the same sequence we run in an agent readiness assessment and an eval suite build.
- Sample traces and build a taxonomy. Read two to five hundred runs by hand. Code each failure: wrong tool, bad arguments, premature stop, runaway loop, unsafe action, wrong answer. Count them. This alone reframes the launch conversation from "it seems fine" to "8% of runs stop early on multi-item orders".
- Write down what correct means for the top three user journeys, with product in the room. Trajectory and outcome, both.
- Turn the top failure classes into cases. Fifty is enough to start. Each case gets an input, a recorded or simulated tool environment, and an expectation.
- Grade deterministically where you can. Argument equality, ordering constraints, state assertions, call-count limits.
- Calibrate any model-graded judge against at least a hundred human labels and report the agreement number. Below roughly 85%, rewrite the rubric or replace the judge. (Here is the full calibration procedure.)
- Put it in CI with thresholds that block a merge. A suite that runs on a laptop when someone remembers is documentation, not protection.
What to expect afterwards
The first honest suite usually finds two or three things the team did not know about. That is the point. It also changes how prompt and model changes get made: from "ship it and watch the dashboard" to "run the suite, read the diff". And it gives the people asking "is this ready?" something better than a demo.
Tracing tells you what happened. Evals tell you whether it should have. You need both, and the second one is the one most teams are missing.