Skip to content
Interface language: English
Audit / Signal

An AI bibliography: a working DOI can lead to the wrong paper

Check whether an AI bibliography’s DOI, title, authors and version identify the same work, then assess whether the material supports your claim.

Article language: English

Illustration: part of a stone column, with a thin line marking a crack.
In this article

Imagine a review in which opening a DOI brings up a real publication, so the bibliography entry gets a green tick. Only later does someone notice that the authors and title belong to a different work. The link did its job, but the review ended too soon. A bibliography requires confirmation that every field describes the same material, followed by reading the material needed to support the conclusion.

A DOI is a persistent identifier for an object, used for scholarly publications and datasets, among other things. The DOI Foundation explains its role in identifying an object and directing readers to it or to information about it. Metadata, the data describing the object, and addresses can be updated without changing the identifier itself. It is a way to locate material, rather than a certificate that every claim placed beside the link is true.

When reviewing material prepared with AI, counting working links is therefore not enough. Establish where they actually lead and whether those are the materials the author used. A bibliography should make the basis of the reasoning easy to find.

One record must describe one publication

A fictional training team is preparing a short report on how people follow assembly instructions. AI suggests eight sources. In one entry, the title concerns the arrangement of illustrations, the names come from a paper on teamwork, and the DOI leads to yet another article. This is an invented review case; we are not attributing the error to a real author or tool.

For an entry like this, preserve the original wording before correcting it. Otherwise, there will be no record of what was submitted. The reviewer separates the title, authors, year, publication venue, material type and identifier, then compares them with the publisher's record or the appropriate registry. A valid DOI prefix does not confirm the remaining fields.

The Crossref documentation describes access to a single metadata record for a DOI registered with Crossref. The data comes from members and trusted sources. The documentation also provides for reporting incorrect or incomplete metadata. A registry helps compare records, but a discrepancy still needs an explanation; it should not automatically be blamed on the model.

A manual review does not require you to build a metadata retrieval program. Open the appropriate record and the publication page. Record which fields were compared and the result. For a process handling many entries, automated metadata retrieval can speed up this step, but it should distinguish a confirmed match from a candidate that needs checking. The first search result is not automatically the work you were looking for.

A missing record in one registry is not enough to reject an identifier. First identify the appropriate registration agency and compare the information on the publisher's website. If an automated tool does not support that route, send the result for review rather than labelling it “publication does not exist”. In the report, distinguish an error in the record from a limitation of the search method used.

Pay attention to the type of material too. A record may describe a chapter, an entire book, a dataset or a separate supplement to an article. In our example, the finding about instructions might appear in a chapter, although the model supplied the volume's title. Adding a working DOI for the book helps locate the broader material, but still does not identify the passage used. The entry needs enough detail to reach the relevant part.

A title should not be freely paraphrased like an ordinary sentence in a report. It may be the key used to find the material. Keep the original title and add an English explanation separately. If the model invents a smoother title, the reader may search for a publication that does not exist under that name, even when some of the other details are correct.

For names, distinguish shortening a list in accordance with the chosen citation style from attributing a work to someone else. An initial, the order of elements in a surname or the rendering of characters may need comparison with the original. Not every difference proves that a work was fabricated. Nor can every difference safely be fixed by changing one letter. Record the cause you have identified or leave the result unresolved.

If the material used has no DOI, do not add the identifier of a similar publication. Keep the details needed to find the right source: its original title, author or institution, edition and a stable address, where available. The absence of one kind of identifier does not justify substituting a different work. Assess whether the underlying material can be traced, rather than rewarding the appearance of the bibliography entry.

Treat the year with the same care. Early online availability and later assignment to a journal issue can provide two legitimate date contexts. Only the correct record establishes what each date refers to. Separating the publication date, event date and date of your own reading prevents corrections based on whichever year happens to be most prominent on screen. The article on publication, event and reading dates (in Polish) explores this distinction.

Distinguish missing material, discrepancies and lack of access

The first of the eight fictional entries leads to material with a different title. That is a confirmed discrepancy between the entry and its destination. The second cannot be opened in the tool being used. That is a lack of access through that route. The third was not found by title and author names in the places checked. That is a search result with a defined scope; it does not yet prove that the publication does not exist.

Do not give all three the status “false source”. That label makes repair harder. A discrepancy requires identification of the right work. A lack of access may call for an authorised route to the text. An entry that has not been found requires further information, or removal from the report's working evidence until the issue is resolved.

Do not look for a replacement merely to keep the total at eight. If nobody knows which material was meant to support the sentence about assembly illustrations, another publication with a similar word in its title does not fix the problem. Return to the claim first and establish what evidence it needs. Then select a source and assess whether the existing wording can be retained.

Aeneia conceptual photography

The problem extends beyond wholly fabricated works. The abstract of the 2023 study by Walters and Wilder distinguishes nonexistent references from errors in descriptions of real materials. It concerns specified historical model versions and a particular sample. We do not use its findings as a measure of a current tool's quality or of a particular company's bibliography.

In preparing the Polish source article, the study's record and abstract were read, rather than the full paper. That reading confirms the publication's identity and the distinction between errors described in the abstract. It does not justify claiming that every methodological detail or results table has been checked.

Your own register should similarly distinguish “record matches”, “abstract read” and “specified passages of the full text read”. Do not call every located item a source you have read. For longer material, the passage's location forms part of the evidence. A page number, section name or table identifier lets the next person return to the basis of a claim without searching the entire publication again. In a company AI search system, a similar boundary separates finding a document from using it correctly in an answer. The article on sources in company AI search (in Polish) discusses that boundary.

You can discuss your bibliography-checking process and the possible scope of support with Aeneia Audit. Assessing the substance of research in a particular field requires someone with the appropriate expertise.

Version and status can change what you can cite

The same topic may appear as a preprint (an early version made available before formal publication), a published article, a conference paper and a later correction. Do not assume that similar titles mean identical findings. Record which version was read and how it relates to the bibliography entry.

A downloaded copy should have an unambiguous version description. Do not rename it in a way that implies a later edition you have not read. If two files have similar names, compare the title page and edition details before combining notes. Notes from one version do not automatically apply to the other. This allows the team to reconstruct which material supplied a particular passage. If a report relies on a table from an earlier file, simply adding the DOI of the later publication does not confirm that the table matches.

When comparing versions, return to the element you use: a number, a methodological limitation or a condition attached to the conclusion. You do not need to describe every cosmetic change to the document. You do need to establish whether the new edition preserves the basis of your sentence. If that has not been established, the register should show the gap rather than automatically marking the newer version as read.

Crossmark helps check a publication's current status, including corrections and retractions. Crossref explicitly states that the mechanism's presence is not itself a guarantee. The information depends on reports being maintained. During review, read the notice you find and assess its significance for the passage being used, rather than stopping when you see the button.

A correction may concern authorship, a number in a table or the interpretation of a finding. Each has a different consequence for the report. A retraction also requires reading the relevant notice. It must not be hidden by removing the label from the bibliography while leaving the same conclusion in the text. First establish what changes in the argument's evidential basis.

If the bibliography will be used over a longer period, assign responsibility for rechecking key sources before the report's next edition. Not every update to a minor footnote requires a fresh study of the entire topic. A change to material underpinning a recommendation does justify revisiting the relevant claims. The task follows from the source's role, rather than a number of months mechanically entered in a calendar.

A consistent bibliography is the starting point for assessing the argument

Suppose that, after clarification, every entry used in the fictional report describes the right publication. The passages concerning assembly instructions still need to be read. A study may concern a different user group, another task or conditions the team does not reproduce. Establishing the material's identity is necessary to check the argument, but does not establish its relevance to a new application.

For each decisive claim, preserve the relationship between your sentence, the relevant source passage and the limits on transferring the finding. If the authors describe an observation in a small sample, do not turn it into a universal design rule. If they compare two versions of an instruction, check exactly how they differed. The word “illustration” in an abstract does not support arbitrary advice about graphics.

Do not infer support from the number of links either. Several discussions may repeat the same material. With conflicting sources, provenance and scope matter more than the numerical advantage of one column. One matching bibliographic record does not become several independent confirmations when links to a repository and the publisher's website are added. The article on conflicting data about Asian markets (in Polish) examines provenance and scope.

When reviewing a market report, the equivalent question is whether the source addresses the decision the company is actually making. Here, we began one step earlier by establishing which materials the bibliography contained at all. Without that, a later review may assess an argument linked to someone else's publication. The guide to reading a Korean market report (in Polish) discusses how evidence relates to a decision.

Corrections to the bibliography must also be traced through the report's text. Changing or removing an entry can leave a sentence unsupported, while renumbering references can assign it to a different work. Follow each changed citation through to the right material. A longer model explanation of its own reasoning cannot replace that connection. The article on AI explanations and independent evidence (in Polish) explores that distinction.

Record the correct publication and what was read, then link each cited passage to the claim it supports. If the evidence is missing, revise the claim.

Aeneia / Aeneia editorial

Editorial article. Source links are included in the text.

About Aeneia

Brands we have worked with

  • L’Oréal
  • Garnier
  • Netflix
  • Carrefour
  • Warner Bros. Discovery
  • Biedronka
  • HBO Max
  • Wolt
  • Żabka
  • COSRX
  • Embassy of China in Poland
  • Mixa
Aeneia / Initium

From knowledge to a decision

Tell us what you want to put in order. We will agree on where to start.

Contact Aeneia

We partner with and use solutions from:

  • Google
  • OpenAI
  • Microsoft
  • Stripe
  • Anthropic
  • Linear