Skip to content
Interface language: English
Audit / Signal

AI compares document versions: the riskiest change may be a deletion

How to review an AI-generated difference report: deletions, units, moves and links back to the right document versions.

Article language: English

A gardener adjusts the water flow at a prototype made from roots beneath a red greenhouse canopy.
In this article

"Minor clarifications and editorial improvements were made." That is the summary of two versions of a display stand specification. Yet an exception concerning the side shelf has disappeared from the new version, and one dimension has a different unit. The document is shorter and easier to read. The account of its changes has turned out to be a little too reassuring.

AI can help organise differences between documents and point out passages for review. To evaluate that use, check whether all changes have been captured and attributed to the correct versions, and assess the significance of the deleted text. Summarising the new document is not the same task as comparing the old and new ones.

An omitted old condition may be invisible in the new version alone. A model can describe everything that remains and still fail to say what is missing. A test therefore needs a known set of changes and a way back to both sources. A polished sentence about editorial direction cannot replace a trail to a specific passage.

First establish which versions you are comparing

Start by naming the documents and the direction of comparison. Which is earlier, which is later and what does each cover? A date in the filename does not always settle the order. Colleagues may send files from different branches of work. When a difference arises during concurrent editing, show the conflict and available versions clearly (in Polish) rather than choosing one file without knowing about the other changes. Establish the versions with the person responsible for the material before the tool begins describing differences.

Keep the input copies and a simple identifier for the run. If a document was corrected after the comparison, the old report should not be presented as an assessment of the latest file. The record must let someone reconstruct the pair used to produce the result. A clear association is enough; there is no need to copy an organisation's entire history into every report.

Also preserve how the input was prepared: export, reading and any division into parts. When there are several parts, you need to know which passages were compared. A moved paragraph can land in a different part of the newer version, so comparing only pairs with the same number may treat it as a deletion and a separate addition. That result requires an assessment of the relationship, not an automatic match based on similar wording.

Check whether the compared scopes are equivalent. A full specification and an exported table excerpt are not an ordinary pair of versions. If one side lacks an attachment, mark that fact. The model should not treat the entire missing file as a deliberate removal of its contents when the export simply did not include it.

For a PDF or scan, assess the extraction. Checking OCR before AI analysis (in Polish) helps find an input error that could later look like an author's change. Reading "mm" as "m" creates a different discrepancy from an actual edit to the unit. The report should distinguish these sources of uncertainty.

Also establish the kind of comparison. A team may need a list of every textual change, differences in meaning, or information about the effect on a catalogue. Those are related but distinct layers. First establish what changed. Only then can you assess what the change means for the task.

Microsoft describes comparing documents in Word (in English), including character-level or word-level changes and a result displayed in a new document by default. It is an example of a tool for identifying differences. Assessing their meaning for a particular specification remains a separate step.

For text materials, the git-diff documentation (in English) describes word-level differences and the default use of whitespace to delimit words. A mechanical record helps locate additions and deletions. It does not decide on its own whether a moved sentence changed the scope of a rule or whether a new unit was an intended correction.

In a test, you can compare the AI result with a difference view suited to the document format. Do not assume that every automated diff is a complete reference for meaning. Table layout, text order and comparison settings may need separate assessment. The reference should itself be checked against your material.

Prepare changes the model should not lose

Take a fictional specification for a stand used at trade fairs. The earlier version describes dimensions, the contents of the set and the scope of the side shelf. While preparing the newer version, the team deliberately makes several changes. Their list stays outside the model's input as an evaluator's answer key. This makes it possible to check what the tool actually finds.

A unit change. The height "84 cm" is written as "84 mm". This is not simply a shorter sentence. In the fictional example the dimension has a different meaning. The result should show both forms and the location, without independently correcting the newer version to a presumably more likely value.

Add an equivalent case too: "24 cm" changes to "240 mm". The characters differ, but the dimension remains the same. A good comparison can record the change in notation and separately describe its meaning. It should not put both examples into a single "size changed" category.

A deleted exception. The final condition disappears from "The side shelf is part of the set, except for the narrow version". The new text may suggest a different scope of contents for the narrow version. The report should show the deleted passage and its relationship to the sentence that remains. Merely saying "the information about the set was simplified" hides a decision that needs to be made.

The product owner must establish whether the change was intentional. The tool should not guess. Removing a condition might reflect a design update or an editorial mistake. The comparison should expose the difference and its possible effect, leaving confirmation to the person responsible for the content.

Aeneia conceptual photography

A moved paragraph. Instructions for setting up the stand move from the end of the document to the opening section. If their meaning has not changed, distinguish the move from a new rule. If the link to a particular version disappeared during the move, assess that context too. Position can affect the scope in which a sentence is read, even when its wording is identical.

An added feature. The new version mentions a pocket for materials. The report should identify the addition and its source, but not add that the pocket is standard for every variant if the document does not confirm it. Check whether the model describes the added pocket within the source's limits rather than extending it to other variants.

Add a table change: the same parameter remains in a row, but its value moves to a different variant's column. A linear text extraction may not retain that relationship. The expected result should identify the specific variant, not merely the presence of a number in the document. Return to the table view and check its headings. This helps distinguish recognising characters correctly from assigning the right meaning.

These cases raise different acceptance questions: was the change detected, was its direction preserved, was the location identified and is the stated meaning supported? A single rating of "correct report" makes the cause of an error harder to find. A tool may find a unit but mix up the documents and attribute the older notation to the newer version.

Also test the absence of changes. A pair of identical versions, or documents that differ only in agreed formatting, can reveal invented differences. The comparison process should be able to find no material change when the answer key contains none. A lengthy report is not a measure of care on its own.

The distinction also matters when generating product descriptions with AI (in Polish). If a new product sheet changes the set's contents, you need to know which public statements require updating. The new description alone will not show every place that relies on older information.

Aeneia Audit is an entry point for an operational assessment of AI use in such a workflow. To define the scope, prepare a pair of fictional versions, the expected differences and a description of the subsequent decision based on the report.

A report should lead to the passage, not just describe the document

For each difference, expect the earlier text, the new text, the location and the kind of change. A page number can help, but may quickly become insufficient when the layout changes. A heading, field name or short reference passage can make checking easier. Choose a method suited to the format and keep it aligned with the specific versions.

Check that quoted passages actually occur in the indicated versions. A model can produce a convenient paraphrase that looks like a source quotation. If the report marks text as verbatim, compare it character by character or establish a way to retrieve it from the document. Call a paraphrase a paraphrase. This tells the reader whether they are seeing source text or an explanation prepared by the tool.

Separate a factual description from an assessment of impact. "The exception for the narrow version was deleted" is different from "All versions now include the shelf". The latter may be a hypothesis requiring approval. The report should preserve that status so another system does not treat an interpretation as a confirmed specification.

When translating documents with AI (in Polish), reviewers check faithfulness between languages. Comparing versions likewise requires care with numbers, negations and exceptions, but the task concerns relationships between changes. A fluent summary of the new text alone does not answer what happened to an omitted old condition.

If a tool cannot access one part, it should say so. A missing attachment is not evidence that nothing changed in the attachment. The limitation might concern a table, a scan or a portion outside the input limit. The team needs to see where the result rests on material that was read and where a basis is still missing.

Check a person's subsequent review of the report. Ask them to locate one change and confirm it in both documents. If that requires searching the entire file, improve the references. In recurring work, the cost of returning to the source can matter more than the time it takes to generate an elegant summary.

When the workflow includes removing data, also assess checks on automated document redaction (in Polish). A comparison can help identify the scope of textual modifications, but it does not itself confirm that every remnant was removed from a complex file. Each stage needs a condition suited to its task.

For archiving company knowledge (in Polish), preserve the link between the report, the input versions and the approving decision. Otherwise, someone may later use a correct analysis of an old pair as evidence for the current document. Accuracy does not prevent a report from being assigned to the wrong material.

Add reversed version order as a controlled acceptance case. The report should reverse the direction of additions and deletions accordingly while preserving references to the sources. This is not intended as the everyday workflow; it reveals a tool that mainly describes one document regardless of the other. The record gives a concrete answer to whether the process compares a pair or merely summarises the material it was given.

After improving the process, return to the known set of changes and add a new document layout. Keep the errors revealed by the earlier test. A later model update will then not be assessed only against the easiest example. The final report should list the formats, scopes and kinds of differences checked.

A good comparison supports a decision by showing what was added, what was deleted and what still needs clarification. It can be brief when the material allows. It should not be so smooth that the deleted exception disappears among sentences about tidying the text. That deletion may be the reason the review was started in the first place.

Aeneia / Aeneia editorial

Editorial article. Source links are included in the text.

About Aeneia

Brands we have worked with

  • L’Oréal
  • Garnier
  • Netflix
  • Carrefour
  • Warner Bros. Discovery
  • Biedronka
  • HBO Max
  • Wolt
  • Żabka
  • COSRX
  • Embassy of China in Poland
  • Mixa
Aeneia / Initium

From knowledge to a decision

Tell us what you want to put in order. We will agree on where to start.

Contact Aeneia

We partner with and use solutions from:

  • Google
  • OpenAI
  • Microsoft
  • Stripe
  • Anthropic
  • Linear