Test an agent before a customer meets one.

Software testing rests on an assumption so old it is rarely stated: run the same input twice and you get the same output twice. Assert on the output, and you have a test.

An agent breaks that assumption on purpose. Ask it the same question twice and it will answer well both times, in different words, sometimes by a different route, occasionally in a different language. Traditional testing does not get harder here. It stops measuring anything.

So agent products get shipped on judgement instead. Somebody tries the thing a few times by hand, it behaves, and that becomes the evidence. It is not evidence, and everyone using it knows that.

Arena is the proving ground that replaces it: a population of synthetic situations — a call, an email thread, a meeting — run against the real agent and graded on what it did, not on the words it chose. Run enough times, that is a number, and a number is something you can hold a release to.

You cannot assert on the words. You have to grade the behaviour. The shift Arena is built around