Hands-on notes on evals, red-teaming and shipping
September 4, 2026 · Judgeworthy
Correctness evals say nothing about an agent that takes forty steps to do a three-step job. Here is how to turn step count, tool calls, tokens and latency into hard assertions, and how to test the loop on purpose.
Read moreSeptember 4, 2026 · Judgeworthy
Injection is not the only way an agent crosses a permission boundary. Sometimes the user just asks. Here is a pytest harness that probes authorization directly and turns every crossing into a replayable case.
Read moreAugust 27, 2026 · Judgeworthy
Hostile text arriving through an MCP tool is the injection path we find most often. Here is a stub server, a case file and pytest assertions that turn it into a suite you can re-run after every fix.
Read moreAugust 25, 2026 · Judgeworthy
Traces tell you what your agent did. Scores turn that into a quality signal you can chart, threshold and argue from. Here is how to add them with the Python SDK.
Read moreAugust 21, 2026 · Judgeworthy
Most agent failures are visible in the tool calls, which are structured data. Here is how to assert on them exactly with Promptfoo and gate a merge on the result.
Read more