AI removes data from a document: a black rectangle is not the end of the check
Test detection, file processing and the final copy for omissions, excessive removals and remaining information.
Article language: English

In this article
The document preview looks fine. A black rectangle covers the name, and another covers the address. Copy the text into another window and the name returns, like a guest asked to leave the sitting room but given a key to the kitchen. Visibly covering information and removing it are different things. Detection is one stage of the work.
Assessment of automatic identifier removal should cover the whole path: input, detected passages, a decision about scope, file processing and checking the result. A report ending with a list of names found has not yet answered what was in the material passed onward.
This article proposes an operational trial using artificial documents. It can reveal omissions, excessive removals and remnants in a file. It does not decide whether a particular item is anonymous in law or may be shared with a particular recipient. That decision needs the appropriate context and responsibility beyond assessment of the model alone.
Decide what must disappear and what must remain useful
Imagine a fictional report about workshop equipment. It contains a person's name, address, contact details and a description of a problem with product K22. The copy is intended for discussion of how the fault was handled without disclosing the reporting person's details. The product code and problem description are still needed.
The team must define the scope. “Remove sensitive data” is not enough because participants may understand it differently. In the trial, record specific categories and examples to remove. Separately identify technical data that must remain and passages needing a person's decision.
Prepare a control version with known identifiers. If AI helps create the set, check consistency of synthetic data and its relationships (in Polish) so mentions in different files actually refer to the same test people and cases. A fictional name may appear in several forms: in a heading, body and attachment. The team should know every location. Without a reference point it is hard to distinguish a good result from one that simply attracts no attention.
Do not restrict the set to the easiest “Name: ...” layout. A report may say “Please pass a collection message to Iga Próbna”. Another mention might concern a copied recipient or a photograph caption. Choose examples suited to the actual document types, using fictional data only.
Check repetition of the same information in different places. A name can disappear from the first paragraph yet remain in a signature and a quoted earlier message. Record all expected occurrences on the control card and assess each location. Marking one entity on a list does not establish that every occurrence in the document was covered. Keep this as a separate test after each configuration change.
Add similar-looking content that must remain. A product code, component name or technical-stage number may resemble another kind of identifier. Overly broad removal can destroy the document's meaning. The recipient may receive text without a name but also without the information needed to understand the fault.
Microsoft's transparency note on PII detection notes possible omissions and incorrect marking of content outside a sought category. For data removal it recommends considering human review. This describes a mechanism's limits; it does not confirm a particular processed document.
Assess the result in two directions. What should have disappeared but remains? What should have remained but was removed? These errors have different consequences and should not disappear into one “accuracy” score. In our report, a retained contact and a removed K22 code need different corrections.
With data minimisation before AI use (in Polish), first limit the input itself. Automatic detection should not justify sending entire folders merely because someone will remove details later. A trial needs material appropriate to its task, not a larger folder for the team's peace of mind.
Detecting a passage is the beginning, not the end of working with a file
Detection can return text ranges, a category and a confidence score. The next stage must correctly apply those ranges to the right document version. If the text changes after detection, earlier positions may no longer point to the same place. Set the operation order and preserve the link to the input.
Check what the tool does: mark a location for review, visually cover it or remove content from an output copy. Buttons may have similar names, but the effect must be confirmed. Acceptance should cover the saved result after all steps, not just a preview of marked rectangles.
Adobe's description of data in PDFs lists metadata, attachments, comments and hidden text alongside visible text. That is a reason to separate checks of visible content from checks of other file locations. A list of possibilities does not prove the presence or removal of data in your document.
In a controlled example put artificial information in a visible paragraph and separately in a location the format permits you to check, such as document metadata. Save and reopen the result. Check whether the process covered both locations as planned. Do not assume a text correction automatically changed every file property.
Try searching for the known fictional name and copying text from the removed area. These useful tests expose some errors, but no match is not a complete assessment of every document layer. Match the check to the format, tool and intended use. More complex material may need separate technical file verification.

Also open the handover package if the result travels with additional files. A correctly processed copy may accidentally be attached alongside the original in the last step. End the trial by checking the package's actual contents and identifying the file intended for the recipient. This is a separate process error that changing a name-detection model will not fix. The owner of the last step needs a clear acceptance condition.
Check the exported result as well. The process may work in one view while passing on a different file from the one the operator inspected. Record which artifact is intended for sharing and where it is. A name such as “anonymised” cannot replace a link to the version actually checked.
If a document needs OCR before AI analysis (in Polish), assess the detection input. A misread letter may change a name or code. A model working on text need not see what remains legible in the page image. The control chain should cover that relationship.
Likewise, generating alternative descriptions of images (in Polish) can introduce text from a photograph into a new field. If the flow includes images and descriptions, check both places. Removing data from a paragraph does not answer what was saved in an additional description.
Check context that can still point to a person
Not every item has a convenient label. The fictional report might retain “the only person receiving parcels at the small workshop near ...” or an incident detail recognisable to recipients. Removing a name alone does not decide whether the text can identify someone. Mark such passages for a separate contextual assessment in the trial.
Do not expect one universal model decision for every sharing purpose. The same detail may be needed to explain the matter internally and unnecessary in a public training example. First name the recipient and task. Only then decide what to retain in the copy.
For artificial material prepare two uses: discussing the repair method and publishing a service example. The team should compare the required information scope. This is not a legal classification. It reveals that “remove names” can be too narrow for the real purpose of preparing the document.
A result with unclear context needs a decision owner. Do not add automatic publication approval merely because a tool found none of the categories on its list. No marking can mean a failure to recognise or lack of support for that type of information. The report should preserve that uncertainty.
Consider continuity of meaning after removal. “Her answer about that component was sent” can become unreadable if all references disappear. An approved edit to the copy may be needed, such as replacing the person with a neutral role. Distinguish this edit from detection and removal of characters.
For larger changes, comparing document versions (in Polish) helps show whether a technical condition or negation affecting handling disappeared with the information. A readable result should retain content needed for its purpose, not merely look safer because of the number of black boxes.
The Aeneia Audit offer can start an operational review of such a flow. To define scope, prepare an artificial document, a list of information to remove and a description of where the result goes.
Acceptance should preserve limits and a way to retest
For each case retain input, expected change scope, tool result and assessment of the final copy. Separate source material from the version intended for further use. An operator should be able to identify the correct file without guessing the order of names ending in “final”.
With a known test set you can count missed and excessively changed passages, but also preserve their type. Several repetitions of one name do not equal several independent types of situation. A number should support the description, not conceal that the trial covered one easy template. When expanding later, add a new format or layout as a separate scope and check it with its own cases.
Record error category and location. A missed name, an uncovered attachment, an incorrectly identified range and a damaged product code need different actions. This helps decide whether to change recognition, processing order, format handling or editorial criteria.
After a fix, repeat the discovered case and neighbouring variants. Better recognition of names should not remove product names without being checked. If a qualification threshold changes, inspect both directions of error. Showing one document with more markings is not enough.
AI data retention and deletion (in Polish) also concerns materials created during checking. The source copy, intermediate result and report can contain different information. Decide their purpose and access rather than treating every file as equally ready to transfer.
Clearly list checked formats, languages, categories and exclusions in the report. A controlled set can reveal a problem, but does not justify an unconditional promise for all company correspondence. A decision to extend use should rest on the described scope and responsibility.
A good process can show what it found, what it changed and what was checked in the result. It can also hold a case requiring another assessment. A black rectangle can be part of the final view. Evidence comes from the entire process matching the agreed purpose, not the neatness of its edges.








