NewDesk team14 min read
Two different mistakes
An assistant can be wrong in two ways. It can fail to recognise that a customer wanted a call booked, and ask one more question. Or it can decide a customer wanted a refund when they were asking about one, and run it. The first costs a question. The second costs a customer. They are not the same kind of error and they are not tested the same way.
How a skill is chosen
When a message arrives, the desk builds a short list of the skills the words could mean, asks the model to choose with the look-alikes shown side by side, asks for a second opinion, and then grounds every value the model filled in against what the conversation actually said. A value that was not said is not used. Finally, for anything that acts, the customer’s own agreement is required before it runs.
The evaluation
We keep a file of scenarios: families of real messages and the look-alike skills they could be confused with, a call booking beside a callback request beside a question about calls. A dry run, with no model, checks that the right skill makes the short list for every scenario. A live run puts the real messages through the whole decision and reports every wrong run and every unwanted one.
The bar
The bar for a live run is zero wrong or unwanted runs. Not a percentage. A miss is allowed, because a miss is a question; a wrong run is not, because it is an action taken on a customer who did not ask for it. When a new kind of skill is added, a new family goes into the file first, so the skill it most resembles is tested against it before it ships.
Why not read the chats
Reading a sample of conversations tells you how the assistant sounds. It does not tell you what it would have done with the hundred messages that were not in the sample. The evaluation does, every time the decision changes, in minutes.