AI text detectors: a percentage on a screen does not sign an author's name
Evaluate what an AI text detector result says about company material, with known sample histories, false alarms and limits on authorship inference.
Article language: English

In this article
“87% AI” looks precise. Forma studio received that result for a text about its new collection and began looking for someone to blame. Yet nobody had established what the number meant: a share of highlighted passages, a classifier score or a probability defined by the provider. The detector supplied a label and the company had almost attached a person's name to it.
Forma is fictional. It is organising acceptance of material from external authors: product descriptions, instructions and website copy. It wants to know whether content was prepared as agreed and whether its basis can be checked. Those are reasonable questions, but a detector does not automatically answer either. A human-written text can contain an error; AI use does not mean a description conflicts with the supplied specification.
Treat the tool's result as something to evaluate. Before adding it to a workflow, establish what claim it can support, for which material and with what risk of error. The following scope concerns company publications and collaboration around them. It does not cover employee selection, penalties, pupil assessment or resolution of disputes over text rights.
First establish the question for the writer
“No AI” can mean very different expectations: writing every word independently, no automatic paraphrase, permitting language editing, or a ban on sending client material to an external tool. Set the conditions out in ordinary language. Otherwise a detector appears to enforce a rule the team cannot explain together.
For Forma it may matter whether the author checked product attributes, retained agreed names and can identify information sources. If the company permits editorial assistance, ask for its scope. One automated flag cannot by itself distinguish correcting a comma, rewriting a paragraph and generating a whole description with incorrect parameters.
Separate origin from quality and data-handling conditions. Check a description against the product sheet using that sheet. Supporting a promise needs appropriate material, as explained in the B2B offer evidence guide (in Polish). A detection result replaces neither check. Do not bundle them into one “text passed review” rating.
Before sending a file to a detector, establish whether it may be given to that particular tool and under which conditions. Use authorised control materials for initial tests. You do not need an unpublished client strategy to test how a result label is understood. Record the product, version or available service date, settings, language and input scope.
What research shows and what cannot be transferred to Polish product copy
In PAWN++ in the Wild, a 2026 paper, the authors compare results on prepared datasets and transformations of web texts. They describe reduced performance under the studied conditions and limit conclusions to the models and procedure used. This is not an assessment of every current detector or a ready-made result for Polish product cards.
An earlier 2023 study by Liang and colleagues found that the tested detectors incorrectly marked some texts written by people for whom English was not a first language. That is a particular trial, language and set of tools at the time. It does not prove the same error rate for every author group and every current product. It supports testing your own type of material.
For Forma the practical conclusion is not to adopt someone else's performance number without the conditions in which it was obtained. A short assembly instruction may repeat phrases because it covers repeated actions. A catalogue uses the same terminology across a series. An orderly and unsurprising company style should not by itself become proof of origin.
Likewise, avoid identifying an author from individual words. “Comprehensive” may irritate an editor but does not carry a language model's identity card. Read the sentence and improve it if it adds no meaning. Assessing naturalness and attributing origin are different tasks. Editorial experience does not turn a good eye into an infallible detector.
Prepare a test with a history you actually know
A control set should contain texts with known production histories. Agree with authors on independently written materials, retained AI-generated texts and versions after real editing. Keep reference labels outside the detector's input. If you take another detector's result as truth, you mainly test agreement between two assessments, not agreement with the text's history.
Select material representative of daily work: a short description, longer guide, list of steps, Polish and a foreign language used by the company. Record length and format. If the provider specifies a minimum text length, treat it as a test condition. Do not append an unrelated paragraph merely to make the tool accept the file; the result would concern different material.
Check file ingestion too. The detector may receive only the main content or text together with a recurring footer, captions and contact details. Compare inputs deliberately, according to intended use. If the result changes after removing a fixed block, record both scopes. Do not attribute the difference to new information about the author when the operator changed the assessed content.
Check the whole and a selected passage separately if that is how the team plans to work. Cutting the introduction or leaving only a feature list changes the input. A whole-document label does not automatically explain the origin of each sentence. Describe a mixed version as mixed in accordance with its known history. Do not force a binary label where the process included several kinds of work.

Record the exact message, not just its colour. Use the product documentation to establish what the scale means and where its classification threshold lies. A high number need not mean the percentage of words written by AI. If the definition is unclear, retain that uncertainty in the protocol. Do not replace it with a convenient explanation invented by the person running the test.
Separate at least two errors. A false flag attributes AI to text known from its production history to have been written independently. A miss is failure to flag text known to have been generated. The company's costs differ: the first causes unnecessary explanations with the author, while the second may pass material prepared outside the agreed process. Overall accuracy alone hides this difference.
Treat no result separately. A rejected file, text that is too short, unsupported language and a service error are not valid “human-written” decisions. Keep those cases and tool messages in the test. If the team routes them to manual review, include that work in the summary. Otherwise a tool can look convenient because difficult materials vanished from the count.
An invented calculation shows false alarms' significance. Among one hundred materials, ten are AI texts and ninety are independently written. The tool flags eight of the first group but also nine of the second. Of seventeen flagged files, only eight actually came from AI. This is not a result for a particular product; it shows why detections without false alarms are insufficient to assess usefulness.
Do not turn a small set into a guarantee of effectiveness for the whole company. Give each group's size and specific errors. If none were found, record the scope of that observation. It does not mean there will be no errors in unknown texts. The result should allow the test to be reproduced on the same files, not merely reassure the reader with a green summary.
Repeat the test after a tool update or a change of language or editing method. Keep the same sample and comparable settings. Principles for evaluating a model used as a judge (in Polish) help ask the right question: how do we check the tool entrusted with evaluating other material? Its confident tone is not evidence of accuracy.
When comparing two products, use the same files and do not select a more convenient subset for each. Set the threshold before evaluation and describe a threshold change as a new variant. If settings were fitted to particular examples, also check an independent sample. A result obtained after repeatedly tuning settings on known files does not directly show behaviour on the next assignment.
The work history provides material for a conversation
If a flag is unexpected, return to the particular file and assignment conditions. Ask for an account of the work, product-attribute sources and available versions within the scope agreed for the collaboration. Do not begin with “the system proved AI use” when all you have is a classifier label. The conversation should resolve checkable questions, not force an author to fight an unexplained percentage.
Consider a description containing a mandatory sentence supplied by the client. The author may have independently written the rest and pasted that passage as the brief required. Establish the scope of that contribution rather than attributing the whole file's origin to one person. Likewise separate the base text from later editing by another editor. In a multi-person workflow, “who wrote it?” requires identifying the version and passage.
Revision history has limits too. It shows certain saved stages, not necessarily the entire process outside the editor. Absence of a detailed trail is not automatically evidence of tool use. Conversely, a smoothly growing version does not establish that every fact was checked. Record what the supplied material actually shows and which questions remain unanswered.
Comparison with AI-image provenance (in Polish) helps separate evidence of a specific event from inference about content. Each kind of evidence has a scope. Do not transfer the meaning of one marker to an entire publication. At Forma, a product owner's approval of the specification helps assess parameters but is not a complete authorship history for every sentence.
Pay particular attention to material presented as customer statements. Synthetic UX research participants (in Polish) do not become real interviewees because of a low detection result. Ask about such statements' origin in the research documentation: who participated, what was actually collected and how the summary was made. A style check cannot replace that record.
Do not let a flag pretend to be confirmation
If a detector result enters a material register, retain the tool name, date, file version and review status. “Flagged by a detector” differs from “confirmed by reviewing the history”. Without that distinction a later team may treat an old prediction as established fact. The problem also appears when AI suggestions return as new data (in Polish).
Decide who can change the status after checking and what happens to the earlier label. The corrected result should remain understandable to people who saw the previous report. Do not send another automated alert based on a label already challenged. A limited trial can follow the whole route: detection, explanation and register update.
Before deciding to use the tool, write down what the result actually helped with. Did it identify material needing checks that the team would otherwise have missed? How many explanations concerned false alarms? You can discuss whether and how to include such an assessment in Aeneia Audit. At Forma, text acceptance still ends with checking the material against its basis and the agreed way of working. The percentage on the screen remains the output of a tool that itself needs evaluation.







