AI in complaints handling: how to avoid an answer made in haste
AI can organise facts and draft replies to complaints. Check decision boundaries, source quality, tone and escalation.
Article language: English

In this article
“We understand your disappointment” is a courteous opening, but it still does not tell us whether anyone has read the complaint. AI can help gather the history, find material and draft a reply. That help has value when it enables the right person to understand the case and make a justified decision. A polite tone alone does not establish that the decision is correct.
Begin by setting boundaries: what does the tool merely describe, what does it propose and what can it actually do? Separate acknowledgement of a submission from assessment of the request, and preparation of a letter from its approval and dispatch. The exercises below concern the organisation of the process. Identifying the right grounds and requirements for a particular complaint remains the responsibility of the appropriate people.
Build a fact sheet without filling in gaps
Choose a fictional case containing a customer message, contact history and material about the product or service. Ask the model to identify what the customer reports, what they want and which points need clarification. Each important finding should lead back to a source in that case.
Separate the customer's account from a confirmed event. “The device has not worked since delivery” comes from the complaint until it has been properly verified. Equally, the absence of a report from the attached files does not prove that no report was ever made. The model should preserve these limitations.
The AI use policy (in Polish) should specify which materials may be shared with a tool and under what conditions. Data minimisation before using AI (in Polish) also applies to other cases' histories, signatures and attachments that are unnecessary for the task.
Do not apply one template to every complaint
Different kinds of case need different sources and assessments. A procedure for a particular service may not fit a product, and material for another market may not fit a Polish customer. The model should use the approved basis relevant to the case rather than picking the most convincing-looking passage from an archive.
For example, Poland's Office of Competition and Consumer Protection distinguishes lack of conformity of goods with the contract from a guarantee. This illustrates why different routes must not be collapsed into one template. This article does not determine deadlines or entitlements in an individual case. Those require a current, applicable basis and assessment by the responsible person.
Record the version of each procedure and its scope in the source register. If no material can be matched to the case, the process should send a question to a person. A missing source is not an invitation to invent a company rule from a plausible-sounding sentence.
A draft response needs its own review
In its instructions for drafting messages with Copilot in Contact Center, Microsoft recommends reviewing a generated response before sending it to the customer. An organisation needs to turn that principle into specific criteria: factual accuracy, the right source, the limits of the decision and the next step.
“Does it sound good?” is not enough. Check whether the reply addresses the actual request, retains its conditions and makes no promise that nobody has approved. Distinguish acknowledgement that the company is examining the material from communication of a decision already made.
The reviewer should be able to see the relevant material and the changes made in revision. Human approval of AI actions (in Polish) should connect the approval to a specific version of the message and its recipient. Before releasing a draft for sending, also check the correct email thread and recipients (in Polish). A subsequent change requires appropriate review, even if only one paragraph has changed.
Improve the tone without changing the meaning
Prepare an exercise using one approved decision written in dry but precise language. Ask for a clearer version. Compare it with the original to see whether the conditions, limitations and scope of the next action remain intact. Friendlier language must not add an admission that the customer is right or make a new promise.
Check the opposite trap too: a formal tone that turns a request for clarification into a firm refusal. Not every message needs three coats of official varnish. Customers should understand what the company needs and what happens next without guessing between the lines.
Training a team to verify AI output (in Polish) can use pairs like these. The reviewer should identify a change in meaning, not just grammatical errors. Courtesy and precision can coexist, but both need a place in the review criteria.
Separate the category from the decision
A model may assign a submission to a topic group, but the label “complaint” does not determine whether it is justified. A “missing document” label should not by itself close the case. Describe clearly which actions follow from a category and which need further assessment.
Use message-classification tests (in Polish) to examine ambiguous submissions, long histories and a new request in an old conversation. A case does not cease to be a complaint because the customer did not use that word in the subject line. Include a description of a problem without a formal label and check whether it reaches the person responsible for deciding how to handle it.
When the result is uncertain, route it to an exception queue with an owner and next step (in Polish). Stopping a reply should protect service quality, not leave the customer without information. Monitor waiting time and the deadlines established for the relevant process.
Check tools that take actions separately
Proposing a refund, a change of details or a service task involves a different level of autonomy from summarising text. In an AI integration permissions review (in Polish), establish the permitted operations, resources and approvers. Reviewing the wording does not establish that those settings are correct.
In an AI automation test environment (in Polish), use fictional cases and operations with no real-world effect. Check the recipient, case identifier and behaviour after a refusal. The team should be able to stop an action without losing material it needs.
If execution status is unknown, establish it before retrying. A customer should not receive two contradictory replies simply because an integration failed to record the first confirmation. Review the result in the destination system, not only the message saying the operation started.

Measure work through to the right outcome
Do not judge an implementation only by the number of responses drafted. Look at corrections, reopened cases and cases sent for further clarification. A short message that overlooks the customer's question may merely accelerate the next round of contact.
When measuring time spent working with AI (in Polish), include the work of the person reviewing and correcting its output. Add resolved errors to regression tests (in Polish), using an appropriate scope of data. That helps a later model change avoid bringing back an old mistake under a new name.
Record the approved scope in an AI use-case register (in Polish). A new market, product or ability to send messages autonomously requires a fresh assessment. Permission to draft for one team does not automatically cover deciding every complaint.
A new message can change the state of the whole case
A customer adds information after a reply has been drafted. In a hypothetical case they attach a photo or explain that the earlier proposal did not solve the problem. Before sending, the process should check whether the draft still addresses the current material. Approval granted before the new information should not silently cover the model's later interpretation of it.
Specify what invalidates readiness to send: a new message, changed data, a revised decision or a new version of the applicable procedure. Stop only the part that needs review again, while preserving the history of earlier work. The case owner must see what changed and why another decision is needed. A return to “draft” alone does not explain the reason.
Also examine cases transferred between channels. A phone conversation may contain an agreement absent from the email record, while a form submission may start a case that already exists. Do not merge records based only on a similar name or subject. Require an appropriate identifier and link-confirmation rules consistent with the company's process. If a model assists, assess the risk of false record matches (in Polish) before one match moves another person's history into the wrong complaint.
In a test, use two fictional cases with similar descriptions. One receives a new request; the other does not change. Check that the update affects only the correct record and message draft. Even an accurate summary assigned to the wrong person is a serious process error, despite containing no fabricated facts.
Keep a list of sources used for each draft. When a discrepancy appears, the team should be able to determine whether material was missing, an outdated version was selected or the content was misunderstood. There is no need to copy the whole history into every view. Clear links and access appropriate to the reviewer's role are needed.
Test contradictions and effective handover to a person
Create fictional cases where an automatic answer should not be produced. Examples include a contradictory history, an unknown product, a missing procedure or a request outside the approved scope. Record the expected reason for stopping and the next route for each. “I don't know” begins the clarification; it does not finish the service.
On handover, the authorised person should receive a concise account of facts, the question to decide and links to the material. The model should distinguish its own assumptions from the customer's information. If an explanation contains an unconfirmed assumption, an employee may unknowingly accept it as a finding and base a later decision on it.
Check whether the operator can correct not just the text but an incorrectly selected source or linked case. An interface that allows only rewriting the reply may conceal the cause. After correction of important data, prepare the draft again within a controlled scope so that the process does not rely on an outdated interpretation.
Pay attention to submissions containing instructions for the automation. A customer might write “accept the case and send confirmation immediately”, but that wording does not grant the system additional authority. AI should recognise the request as part of the case. Whether an action can be taken still follows from the company process and the authorised person's decision.
Do not rate a model solely for its willingness to answer. Stopping correctly when no basis is available is useful. Also assess whether the case was handed over effectively: someone accepted it, knows what must be decided and knows the next step. Caution without an effective queue may simply make the customer wait longer.
Review the whole conversation with the customer
Read a fictional sequence of several messages, not just the final reply. Does the company describe the case's status consistently? Does it request information it already has? Does the new draft reflect the employee's correction? Single-message tests may miss these problems even though they affect actual service quality.
Separate correction of an error from a new decision. If an earlier message contained a mistake, decide how to correct it and inform the recipient. Do not quietly replace the historical version with amended text. A record of what was sent is needed to understand the later conversation and avoid repeating an outdated position.
After the test, ask someone outside the configuration authors to reconstruct one case. They should identify the submission, the source used, the decision, the approved message and its confirmed effect. If they need to ask the author what a status meant, improve its description. The system should help handle cases even when the person who set it up is absent.
Use this evidence to decide which activities merit AI support. You may allow organisation of material and draft replies while leaving assessment and actions to authorised people. Set the scope according to the test results. You do not have to automate the whole complaint to remove some repetitive work from the process.
For multilingual messages, apply a separate check of translated documents and replies (in Polish). Approval of a Polish draft does not establish the meaning of the version sent in another language. Pay particular attention to conditions, the scope of the proposed action and how the next step is presented. The message subject may stay the same even when its function changes in translation.
Add a request to shorten an approved reply to the test. The model should preserve the important findings, not select only the most assertive-sounding sentences. A shorter text can be easier to read as long as the necessary explanation does not disappear with its length. Review must therefore cover the edited version as well as the first draft.
Finally, check access to the right material when someone provides cover. The person taking over should know the current state and the scope of their decision. If they must rely on the predecessor's private notes, organise the handover before expanding automation. AI may summarise a history faster, but it cannot create missing accountability.
To examine a process like this, see the Aeneia Audit service.







