Test the trajectory, not just the final answer

An agent can reach the right answer the wrong way. If you only check the final answer, you'll miss it.

Picture three runs that all end with the correct output. One took 6 tool calls. One took 40. One overwrote a file it had no reason to touch before it got there. A final-answer check marks all three as passing.

So test the trajectory too:

1. 1Count the steps and the cost per task, and set a ceiling.
2. 2List the tool calls that must never happen in a test, such as writes outside the working directory, and fail the run if one appears.
3. 3Check that tools were called with sensible arguments. A tool that returned something may still have been given the wrong input.
4. 4Run each case several times. One pass can be luck.

You don't need a framework for this. Log every tool call with its inputs, then write assertions over the log.

The final answer tells you whether it worked this time. The trajectory tells you whether it will keep working.

What's the one tool call your agent should never make in a test run?