Task success
Trajectory quality
Cost & latency
Safety & boundaries
Measures
Did the agent achieve the outcome?
Binary success wherever you can define it. Graded scores invite drift in the grading itself.
Requires
An outcome definition sharper than teams usually have
Which is why success definition belongs in the design phase, not the evaluation phase. A team that cannot articulate a good outcome will build a suite that measures something adjacent and reassuring.
Blind to
How the answer was reached
An agent that got the right answer by calling a destructive tool three times and getting lucky scores as a pass.
Measures
Which tools, in what order, how many times
Were any calls unnecessary or destructive? An agent that solves the task via a wasteful or dangerous path is a latent incident.
Requires
Labelling the decision and the path, not the output text
Store the expected trajectory: lookup_supplier → validate_totals → approve, and not create_supplier. Checkable deterministically, and it survives model upgrades that change phrasing.
Catches
Right answer, wrong process
The layer most teams skip entirely — and the one where production incidents most often originate.
Measures
Tokens consumed, tool calls made, wall-clock time
These are correctness properties, not performance nice-to-haves. Treat a regression here exactly as you treat an accuracy regression.
Threshold
An agent at 4× modelled cost is not shippable
Neither is one taking ninety seconds where users expect five, regardless of accuracy.
Watch for
Non-determinism in the distribution
Two runs of the same task can differ several-fold. Report the distribution, not a single figure — the spread is the engineering signal.
Measures
Did the agent respect its autonomy borders?
Did it escalate when it should have, refuse what it should refuse, and attempt only what it was scoped to attempt?
Requires
Adversarial inputs, not representative ones
Structurally different from the other three layers. Prompt-injection corpora belong here, asserted on trajectory rather than output.
Rule
Never delegate this to an LLM judge
Boundary adherence and destructive-action correctness must be checked deterministically. A judge is a quality instrument, not a control.
Golden dataset composition
Representative happy path
Known edge cases
Past production failures
Adversarial
100–200 well-chosen cases outperform 2,000 scraped ones. The value is coverage of failure modes, not volume. The last two segments are what make the suite improve over time — every production incident becomes a permanent test case, which does more for reliability than any tooling choice.
Why agents regress without a code change
The provider updates the model
Behaviour shifts underneath a frozen deployment. Pin production versions and test candidates deliberately.
A tool's response schema shifts
Upstream change, no diff on your side.
Retrieved documents change
The corpus moves under a RAG pipeline.
Run on a bare schedule
An eval triggered by nothing but the calendar is the only thing that distinguishes "our code broke" from "the model changed."