Agent Evals

Definition

Agent evals are an automated test suite for agent behaviour: a curated set of representative cases (a golden dataset), the expected outcome for each, and a scoring method that produces a number the agent must clear before it ships.

An agent without evals is a system whose quality you are asserting rather than measuring. That distinction is what separates agents still running after ninety days from agents that were quietly switched off.

What goes in a golden dataset

• The boring majority. The cases that make up most of the volume, where a wrong answer is expensive precisely because it is frequent.
• The known-hard cases. Split shipments, partial refunds, contract pricing, anything where a competent human hesitates.
• The adversarial cases. Prompt injection attempts, out-of-scope requests, and inputs designed to make the agent exceed its remit.
• The cases that already broke. Every production incident should end with a new eval case. This is the single highest-return habit.

Fifty well-chosen cases beat five hundred generated ones.

The pass bar

The number matters less than the fact that it is agreed before the results are in, and that failing it blocks deployment. A pass bar renegotiated after a bad run is not a pass bar.

Set it per agent and per action class. A support agent answering WISMO questions and a margin agent proposing price changes should not share a threshold, because the cost of being wrong differs by orders of magnitude.

Where they run

On AWS, AgentCore Evaluations runs the suite as part of the deployment path, alongside traces from AgentCore Observability. The important property is that it is in the pipeline, not a notebook someone runs when they remember.

Limits worth stating

• Evals measure what you thought to test. They are a floor, not a guarantee. Novel failure modes arrive in production first, which is why observability and the incident-to-eval habit matter.
• Scoring open-ended output is genuinely hard. Exact-match works for structured decisions; for free text you are choosing between a rubric, a model-as-judge (with its own bias), and human review of a sample. Each has failure modes; none is free.
• A passing suite does not mean the agent is safe. Safety comes from the tool catalog and authorization. Evals tell you whether it is good.

Related terms

AI agent · Tool catalog · Human in the loop · Amazon Bedrock AgentCore