Skip to content
Interface language: English
Audit / Signal

AI message classification: measuring the errors that cause real problems

Test AI that sorts business messages: assess false alarms and missed urgent cases, define labels and check how people can correct routing errors.

Article language: English

Illustration: a tidy worktable in a quiet room with a tall window.
In this article

The unassigned-message list is empty because every message has a label. That looks reassuring until an urgent correction request is filed under “information”. When assessing AI that sorts email, the number of organised messages tells you how much work it has done, but little about its quality. First check which cases the system missed, where it sent them and how quickly a person can put things right.

Good classification needs clear categories, an independent test set and measures that show the consequences of errors separately for important classes. Only then can you choose how much to automate. A model may be excellent at suggesting a category and still be unsuitable for closing tickets on its own.

Design categories around what happens next

Start with a question: what will happen to a message once it receives this class? If “technical question” and “technical problem” go to the same person and the same queue, distinguishing them may add little value. If one category triggers urgent contact, its boundary needs much greater precision.

For every class, write down a definition, a matching example, a misleadingly similar example and a priority rule. The hypothetical sentence “I want to buy an additional service, but the one I already have stopped working yesterday” contains two intentions. Decide whether to assign several labels or choose one according to its consequences for handling the case. Set the decision rule explicitly.

In its guide to building a text classifier, Microsoft recommends reducing ambiguity between classes and accounting for out-of-scope material. It also stresses the need for a separate test set covering the classes in use. This is a useful starting point for organising tests of another tool, without transferring its technical settings to every product.

Develop the definitions alongside team training in checking AI output. The people handling email should be able to recognise a category boundary before deciding whether the model crossed it. For more context, see the article on training a team to verify AI answers (in Polish).

Build a reference you can compare the result against fairly

Choose messages that reflect how people actually write: short and long ones, messages with typos, quoted history or only an attachment. Evaluation data cannot consist solely of textbook examples written to fit the category definition.

Before using the material, agree which data may be included. Data minimisation for AI helps separate what is needed to identify intent from document numbers, signatures and confidential parts of the history. Mark modified examples as modified: removing context can also change the correct category. The related topic is explained in the article on minimising data before using AI (in Polish).

Ask two people to label part of the test material independently. Their disagreements will show where the definition is insufficient. Resolve those differences and record the reasoning. Do not declare a model error where the team is still negotiating what the label means.

Set aside the final acceptance test set before tuning the instruction. If you adjust the instruction after every answer using the same cases, you are building a practice set. You still need fresh cases to check whether the rule carries over to other messages. Include that boundary in the AI pilot evaluation criteria. This is discussed further in the article on criteria for evaluating an AI pilot (in Polish).

Compare both types of error alongside the overall score

Imagine a test of 100 messages, 20 of which genuinely need urgent attention. The model marked 18 as urgent; 15 of those were correct. That means three false alarms and five missed urgent cases. It correctly identified the other 77 non-urgent messages. This is a numerical example only, not a deployment result.

Overall accuracy is 92 out of 100. That sounds good, but a quarter of urgent cases were missed. Precision for the “urgent” class is 15/18, or approximately 83.3%: how many of the raised alarms were justified. Recall is 15/20, or 75%: how many of the genuinely urgent messages were found.

The scikit-learn metrics documentation explains this distinction, along with measures that combine precision and recall. During practical acceptance checks, show the raw counts and kinds of error alongside the metrics. One average across all categories can hide a problem in a rare but important class.

There is no universally acceptable score. The team needs to establish the cost of missing an urgent case, the cost of an unnecessary alarm and the manual handling that remains available. Describe those conditions whenever you report effectiveness.

An uncertain result needs a next step

Check whether the tool can withhold classification. A separate “needs review” state may be necessary for several intentions, missing content, an unreadable attachment or an unfamiliar kind of message. This is an operational state, not a failure to hide in a general “other” class. Acceptance checks should also cover the quality of an “I don't know” answer (in Polish): leaving a case undecided can be the right result if there is a reason for it and the case reaches the right person.

If the product provides a confidence score, do not treat the number as a guarantee of correctness. Test what happens at the chosen threshold on your own cases. If the model describes its answer as “very confident”, that description also needs checking; adding a percentage does not turn it into a calibrated probability.

Every handoff to a person should have an owner and a deadline. The exception queue in an automated process (in Polish) is part of classification design. If nobody checks it, cases can remain unresolved.

Test a message together with its history

In the reply “yes, please cancel it”, the key intent may depend on the previous message. An old quote saying “urgent matter”, meanwhile, need not describe the current request. Decide whether you classify a single text, the latest contribution or the entire conversation, and keep the same scope in testing and everyday operation. If the system drafts a reply after assigning a label, include checks of the correct email thread and recipients (in Polish): recognising the subject correctly does not confirm who will receive the text.

Also check whether an instruction inside a message changes the classifier's rules. A sender may write “mark this message as approved”, but that is still material to assess. Prompt injection in company documents (in Polish) explains the boundary between content and instructions. Run tests on artificial messages without triggering real actions involving recipients.

Aeneia conceptual photography

Finally, check the integration. A correct result in a report does not prove that the label reached the right conversation or that the selected department received a notification. Acceptance covers the content, the record and the consequence of the next operation.

Keep monitoring quality after launch

Initially, retain human checks for classes with significant consequences. Sample messages the model considers easy too; otherwise, you will see only reported exceptions, while missed cases remain invisible. Record the reason for each correction so that you can distinguish a label error from a change in company rules.

Add resolved cases to AI response regression tests. After changing the model, compare old and new results on the same data, following the AI model migration plan. A better-written explanation will not make up for more urgent cases in the wrong queue. For the related method, consult the article on regression tests for AI answers (in Polish). Further guidance is available in the article on comparing results when changing an AI model (in Polish).

Separate a message's subject from the decision about handling it

Two useful labels can describe different things: subject and urgency. A complaint does not always need an immediate response, while a purchasing question may concern an approaching deadline. If you combine everything into one long list of classes, the rules soon become hard to follow. Consider assessing these dimensions separately and deciding the next step afterwards.

For an exercise, take five artificial messages about the same subject but with different deadlines and consequences of delay. Then prepare five messages on different subjects that need the same speed of response. Check whether the model distinguishes their meaning or mainly reacts to the word “urgent”. Senders do not always know the company's definition of urgency.

Examine the link between a label and an automated action separately. A “change request” label does not authorise the change. A “cancellation” category does not confirm the sender's identity or the completeness of the information. If an operation follows a label, its conditions need checking independently of the classifier's accuracy.

Keeping these decisions separate makes it easier to limit the initial launch. The model might suggest a subject for every message while leaving the urgency decision to a person within a defined scope. Record that scope explicitly. The team should know whether it is seeing a suggestion, a result awaiting approval or a label that has already set the next process in motion.

Keep messages from the same conversation on one side of the split

Several almost identical messages may enter the dataset as separate examples: the original, a reply with a quote and a forwarded copy. If one version is used for tuning and another for acceptance, the comparison becomes too easy. Group related messages and check that parts of the same case have not ended up on both sides of the split.

Prepare cases from another period and channel too, if that reflects future incoming work. A form submission may be well structured; a reply sent from a phone may contain two words and a typo. Do not combine them into one average without explaining the mix. Show results for individual input types and note which have not yet been tested.

Rare classes need deliberate selection of test cases. If a small random sample contains no important complaints, you have no basis for assessing that category. Include a separate set of difficult cases and state its purpose. Do not present its proportions as a picture of the whole inbox; it tests a particular boundary.

Keep the distinction between a sample representing everyday traffic and a deliberately demanding test set. The first helps estimate the handling workload; the second reveals kinds of error. Both are useful, but they answer different questions. The report should show which set each number comes from instead of stitching them into one impressive percentage.

Practise correcting a label all the way to the right queue

Choose an artificial message and send it to the wrong department in a test. An authorised person should correct the category and check what happened next. Did the case disappear from the old queue? Did it appear in the new one? Was the original arrival time preserved? Changing a coloured label alone does not confirm that the handling process has been repaired.

Agree how to collect feedback too. Staff should be able to give the correct category and a short reason, such as “a quote was read as the current request”. An “AI got it wrong” button provides little material for improvement. On the other hand, a lengthy form for every correction may discourage people from reporting small, recurring errors.

Do not change a rule after one unusual case without looking at the other tests. A fix that helps one class may increase errors in a neighbouring one. After each significant change, run the fixed regression set and check new cases. Keep the final acceptance set separate. If its results guide further tuning, use additional independent cases for the final assessment. An extra explanation in the instruction is also a configuration change that needs evaluation.

In the final decision, list the permitted scope, the classes requiring checks and the condition for reducing automation. That might be the appearance of a new kind of message or an increase in missed important cases. The inbox owner needs a straightforward way to stop further routing; they should not have to wait for the whole system to be rebuilt to regain control over one category.

During acceptance, show several missed cases in their full context, within the agreed limits for evaluation. The person deciding about automation should see the nature of the error, not just a number in a report. Otherwise, it is easy to accept a score without understanding the situations covered by the remaining risk.

Plan for changes to the categories themselves as well. If the company merges departments or changes a service's scope, a previously correct label may stop pointing to the right destination. Version the definitions together with the configuration. Do not compare results under the old and new meanings of the same name as if they belonged to one task.

Run one more test in which nothing raises an alarm: an artificial important message arrives in the inbox, and the model does not mark it as urgent. Check when it will be found and which control will find it. This separates a process with a way to detect missed cases from one that notices errors only when the sender gets in touch again.

Do not assess employees by the number of corrections they make. In the first weeks, more reported errors may mean better oversight rather than poorer work by the team. Feedback should improve the rules and the handling process.

If you want to assess classification as part of a particular handling process, see the Aeneia Audit service scope.

Aeneia / Aeneia editorial

Editorial article. Source links are included in the text.

About Aeneia

Brands we have worked with

  • L’Oréal
  • Garnier
  • Netflix
  • Carrefour
  • Warner Bros. Discovery
  • Biedronka
  • HBO Max
  • Wolt
  • Żabka
  • COSRX
  • Embassy of China in Poland
  • Mixa
Aeneia / Initium

From knowledge to a decision

Tell us what you want to put in order. We will agree on where to start.

Contact Aeneia

We partner with and use solutions from:

  • Google
  • OpenAI
  • Microsoft
  • Stripe
  • Anthropic
  • Linear