AI email replies: a polished answer can land in the wrong conversation
Review AI email drafts for the right thread, sending account, recipients and draft version before the next action.
Article language: English

In this article
Two messages have the subject “Installation date”. In both, a customer writes politely and asks for confirmation of Friday. One concerns a small exhibition in Gdynia, the other a stand in Wrocław. The assistant prepares a neat reply, chooses the right date and ends with “See you on site”. Unfortunately, in this conversation “on site” means the other city.
This is a fictional mistake that proofreading the paragraph alone would not catch. The quality of an email reply also depends on which message it is attached to, which mailbox will send it and who will receive it. The wording can be flawless while the operation is still wrong.
Suppose the Most team wants to use AI to draft replies to organisers' questions. An employee approves sending. The feature is assessed with test accounts and materials, without sending messages to real customers. The team checks the draft's content, its link to the conversation and its recipients. The cases below form a plan for that assessment; the results for the team's own solution still have to be collected.
A subject helps find a conversation, but does not identify it
“Reply to the email about the date” leaves too much room for a choice. In Most's sample, both conversations have the same subject, a similar date and a similar opening. The assistant should receive an unambiguous indication of the message on which to base its reply. The person checking the result retains its identifier and its connection to the thread, the chain of messages in the same conversation.
Do not replace that connection with a project name guessed from the content. A case number in the company's system may help a person, but the integration still has to identify the right message in the email service being used. During assessment, compare both values: the business case and the technical target of the operation. One correct identifier does not repair the other.
When the selected message is unavailable, the tool should show that it is missing and ask for the material to be selected again. Choosing another conversation with a similar subject on its own changes the task. In the sample, remove access to one fictional message and check whether the process stops there instead of preparing a plausible-looking reply in another case.
The Gmail API threading documentation sets out the conditions for adding a draft or message to a thread: a threadId, appropriate References and In-Reply-To headers, and a matching subject. A matching subject alone does not replace those conditions. This describes the Gmail API mechanism, not a universal instruction for every email system.
Most's report should therefore identify the source message and the target conversation. An everyday preview for the employee can use a readable summary: project, date of the latest message and sender. When in doubt, the user opens the full context. They should not have to read a string of technical characters to recognise whether they are replying to the organiser in Gdynia.
A test with similar subjects should require two different, predetermined answers. If both drafts say “confirmed”, a mix-up may remain invisible. In one fictional conversation, let the confirmation concern nine o'clock; in the other, the employee should ask for the unloading location to be supplied. This reveals whether material from one case has leaked into the other.
“Reply” and “reply all” are two different tasks
In the first case, Most replies only to the person handling the matter. In the second, it includes the specified coordinator. In the third, the original message has been forwarded internally with the question “How should we resolve this?”. A reply may be intended for a colleague even though the quoted text contains a customer's address.
Have the team explicitly choose the intended operation. “Prepare a suggestion for me”, “create a reply to the sender” and “create a reply to all participants” must produce distinguishable results. The word “reply” does not settle whether addresses in the Cc field should be copied again. The model should not expand the recipient list on its own merely because someone is mentioned in the text.
For forwarded correspondence, check the quoted material too. An employee's internal note should not reach an external draft simply because it sits beside the customer's message. Prepare a sample with a clearly marked fictional internal note and a separate question from the organiser. Assess the reply and the quotation included with it, not just the newly generated sentences.
Microsoft Graph describes createReplyAll as creating a reply draft that is sent in a separate operation. The documentation also points to replyTo when the original message specifies a different reply address. An integration must use that mechanism correctly; the displayed sender name is not enough to assess the recipients.
Include the difference between a display name and the actual address in the sample. Two people may both appear as “Coordinator” while representing different organisations. The preview should let the employee inspect addresses and the roles of the To and Cc fields. If the tool also uses Bcc, the hidden-copy field, that scope needs a separate check before the feature is approved. Do not infer it from an ordinary list of conversation participants.
The employee should see the recipients stored in the prepared draft. A list the assistant added in an explanation below its reply may describe a different set. “Reply prepared only for the organiser” is a statement; the checked set of addresses in the operation shows the actual intended recipients. Compare the two during assessment.

Add a test using the team's shared mailbox and an employee's personal account if both occur in the planned workflow. The same text and recipient set do not establish which account will send the reply. Show the actual sending identity in the preview and record the expected one in the test key. A change of account should be visible before the operation is assessed.
A draft has a version of its own
Suppose an employee checks the text and recipients, but a recipient changes before the draft is approved. An earlier approval should not silently cover the new operation. Approval needs to be tied to a specific version: the content, source message, sending account and complete set of addresses.
This is a particular case of human approval of AI actions (in Polish). The preview must show what will be done. An approval button beside polished text does not give the employee control over an operation's invisible target.
For Most, a useful test sequence is to create a reply draft in one conversation, read it back from the system in use, compare its fields with the test key and then change one significant element. Check whether the application recognises the change and presents the current state for assessment again. The test recipient remains a controlled team account, not a customer.
If the draft includes an attachment, mark it with the case number and compare it with the prepared sample. Two fictional quotations can have the same filename but different contents. Also decide whether the tool should attach a document, share a link or leave only the text. Choosing another way to provide it requires checking the recipient's access, not merely the wording “attached”.
Check manual edits to the text separately. An employee may remove an uncertain promise about a date, only for the assistant to restore the earlier sentence when it next prepares the draft. The preview should clearly distinguish the assistant's suggestion from the manually corrected version. This relates to AI and a person changing a record at the same time (in Polish): the team must decide which state was the basis for the work and which one should be retained.
With Aeneia Audit, a particular message workflow can be the starting point, from selecting a case through preparing a draft to responsibility for sending. This makes it possible to discuss an action that can be checked, rather than only whether the model writes in a pleasant tone.
A new message can make a good reply obsolete
In the morning, Most received a question about whether installation was possible on Friday. The draft is ready, but before approval the customer adds: “Plans have changed, please hold off on confirming”. A reply to the earlier state of the conversation can be linguistically correct and out of date when the action is taken.
In the test, check whether the tool recognises the state of the thread again before the next step. If the integration cannot detect such a change, define a limit on the workflow and require a manual check. Do not show the employee an old draft as a reply to the latest message. The label “ready” should refer to the specified material from which the suggestion was prepared.
In another run, the new message changes only the Cc recipients. This separates checking the text from checking the whole operation: the proposed sentence may still fit, but the previous set of addresses no longer does. The team should deliberately determine which message it is answering and who should take part in that reply. Selecting the last text in the conversation alone does not settle both questions.
An event announcing a new email can also reach an automation late. The automation may then start work on an old trigger even though a later part of the conversation is already waiting in the mailbox. Checking delayed events (in Polish), helps separate an old trigger from the current state of the case. With email, that difference can determine whether the entire reply still makes sense.
It is also worth testing a handover to another employee. The new person should be able to tell whether a draft is awaiting review, has been changed or whether the message has already been sent. Marking a request “handled” merely because text was generated can hide the absence of a reply. With AI message classification (in Polish), the category and the subsequent handling status need separate checks.
Assessment ends with the result of the operation
For an implementation that includes sending, an authorised person conducts the test in a controlled environment. Saving a draft does not prove it was sent, and confirmation that an operation was accepted does not yet mean every intended account saw the correct message. Agree with the provider what evidence is available at each stage and how an uncertain result is handled. In a test on controlled accounts, compare the message on the sending and receiving sides, including its link to the conversation. A screenshot of the draft taken before the operation is not enough.
Do not send again just because the screen takes a long time to show a response. First establish the state of the previous operation. Keep the case, draft and execution identifiers and the actual sender in the report. Choose the level of detail according to the control needed; copying the entire correspondence into another report is not required by default.
If the sample reveals material being directed to an unintended recipient, record the exact mechanism of the mistake and stop that way of working until it is explained. A real incident also needs the agreed response to a data incident in an AI tool (in Polish). Changing the model's instruction does not replace establishing what was actually shared and with whom.
Most's final record should distinguish four results: the right conversation was selected, the content was prepared correctly, the right recipients were established and the agreed step was carried out. This lets the team accept writing assistance alone even if it does not yet permit automatic sending. Each decision has its own scope. None can be inferred from a polite “regards” at the end of a draft.








