Step-by-step AI explanations need independent evidence
Check AI results against inputs and an independent calculation rather than treating a fluent explanation as evidence.
Article language: English

In this article
"We have twelve parts. A set needs three. We can prepare four sets." This is a fictional AI explanation for furnishing a room. Twelve divided by three is correct. The conclusion may be wrong because a set needs two stands and one beam, and the stock may not have those proportions.
Adding "I analysed the availability of every component" does not resolve the problem. For acceptance, return to the number of each part and the assembly rule. An explanation can help someone understand a proposal, but fluency is not independent evidence. A good description of a wrong calculation still needs correction, even with five numbered steps and a teacher's calm tone.
Two questions not to combine
First: does the result answer the task? Second: does the explanation faithfully describe how the system obtained it? You can verify a simple calculation without establishing the model's internal process. You can also receive a correct result with a justification that omits an important influence. These are separate subjects of assessment.
NIST's report on explainable AI principles distinguishes an explanation's comprehensibility from its correctness and from the decision's correctness. A correct decision does not establish that the account of how it was reached is faithful. This distinction helps with acceptance without attributing human intentions to a model.
An explanation for an employee may be short: "The number of sets is limited by the available stands." Alongside it, provide counts and a rule for checking that conclusion. Someone preparing equipment does not need a whole narrative of considered possibilities. They need a basis they can reproduce and relate to the actual order.
A researcher asking about the model's internal process has a different task, requiring specialist methods and a defined object of study. Simply asking an assistant to describe more fully what it did does not create such a measurement. Nor should a short user explanation be equated with a complete record of every process in the system.
Here, chain of thought means the presented sequence of reasoning steps. In Lanham and colleagues' 2023 study, researchers intervened in that sequence, including by shortening it or adding errors, and observed changes in answers. They reported differences between tasks. The findings do not support a universal claim that every presented step is always irrelevant.
The practical conclusion for our review is narrower: do not replace checking a result with judging the length of its justification. An explanation can identify a point to check, reveal an assumption or help frame questions. Those uses remain valuable even without grounds to treat the text as a complete record of what caused the answer.
Twelve parts, three different results
For a fresh check of this fictional example, an independent calculation was reproduced in Node.js on 3 November 2026; the code and result of this new run were saved. Whether the earlier run described in the historical version took place has not been established. The fictional rule was that each set contains exactly two stands and one beam. We seek the greatest possible number of complete sets. The program enumerated whole numbers of sets and checked both availability constraints. No AI model or stock system was called. The assistant statement above is invented.
In the first variant there are four stands and eight beams. Two complete sets can be assembled. They consume four stands and two beams, leaving six beams. Twelve parts really are on the shelf, but six no longer have the necessary companions to form further sets. A calculation using only the total lost the structure of the order.
You can check two sets are feasible from the consumed parts. To confirm this is the maximum, rule out a third set: it would need six stands, while only four are available. We have both a feasible assembly and a specific constraint preventing an increase. "Two sets can be assembled" alone would not explain why three cannot.
This matters when AI recommends the best option. Checking one feasible solution does not confirm it beats every alternative. The scope of evidence should match the answer's words. If only feasibility was checked, call it feasible. "Maximum" or "best" needs additional justification against the criterion the team actually chose.
In the second variant eight stands and four beams make four sets, with nothing left. The total remains twelve parts. In the third there are six stands and six beams: the result is three sets and three beams left over. The composition changed, not the total count.
The new reproduction preserves the inputs, rule and results of all three calculations. This evidence answers only the fictional assembly question. It does not establish the state of any real stock, technical compatibility of parts or model behaviour. It shows how to obtain a checkable result outside the assistant's story and limit your own claim.
If a model gave four sets for all variants, we would have a specific error to correct. If it gave the right numbers, that would not yet prove every sentence of its explanation faithfully described its internal process. We confirm agreement with these cases, not an answer to a question the method did not study.
Real processes add the meaning of inputs. Stock figures may include reserved or unusable parts. A correct calculation on unsuitable figures still gives a bad basis for action. First establish what "available" means and who is responsible for data. Do not make an AI explanation confirm a state the system could not access.

Choose a decision, name its bases and establish which evidence confirms the result. In a conversation with Aeneia Audit, you can describe your needs and ask about the possible scope of cooperation. A discussion of "reasoning quality" becomes a discussion of a particular rule, data and how to check them.
Ask for a basis you can read outside the explanation
For a calculation, ask for inputs, units and the rule. For a sourced fact, ask for the relevant passage and document version. For a tool action, ask for its actual result. These differ from "describe your thinking in more detail". Each leads to material that can be compared with the assistant's claim.
Keep the link between input and answer. If the number of stands changes from four to eight during a correction, the new result does not explain the error for the old case. Mark the changed data and recheck the appropriate variant. A reader should be able to tell whether the method was corrected, a gap filled or simply a different task solved.
Do not let the model's story become the input to an independent recalculation. If the assistant mistakenly changes four stands to eight, a calculator can accurately compute the wrong input. Your check starts from approved material. Only then compare the values carried into the calculation with it.
You need not demand a large register for every small answer. Here the counts of two types of parts and the composition of a set suffice. If a conclusion uses several sources, record each source's role. A long list alone does not show where the constraint came from. Checking bibliographic identity (in Polish) also comes before judging whether a cited work supports the claim.
When an assistant describes an action, link its report to the action's trace. "I calculated every item" needs defined input and output. "I checked the file" should make clear what was read and which version. For code tests (in Polish), describing an expected test does not replace running it.
A changed explanation does not approve the earlier result. Suppose that after a comment about proportions the model adds the stands constraint but leaves the number four. The text sounds more mature, but the task is still unsolved. Check the result and remaining parts again rather than accepting a correction merely because the model agreed.
The reverse also happens: the model changes the number to two but keeps the sentence about dividing twelve by three. The result fits the first case, but the justification does not lead to it. Put the answer in order before giving it to an employee. They should understand which rule limits assembly so they do not repeat the error with the next stock level.
When data is missing, a helpful answer identifies the missing field. Instead of "I am not sure", we need a question about the available number of stands. A declared AI confidence percentage (in Polish) cannot replace that information. Even a high percentage does not create a missing inventory record or definition of availability.
An influence can exist without being described
Anthropic's 2025 account of its research examined whether models disclosed using hints that affected answers. They did not always do so. The authors note limitations, including artificially constructed hints, multiple-choice questions and the specified models. It is not a measurement of every current business process.
We do not infer from this why a particular assistant got assembly wrong. Initially a report can say: four sets were given, but with four stands at most two are possible. That identifies the discrepancy and rule to recheck. Explaining the cause is a separate step, not a condition for recognising an evidently wrong number.
Use similar caution when checking with a second model. Agreeing justifications may repeat the same assumption about total parts. An evaluation of a model used as a judge (in Polish) needs a standard appropriate to the task. A second elegant account does not turn the first into an independently confirmed calculation.
When preparing a trial, isolate the difference studied and keep the other conditions. Our three variants have the same total but different proportions. This lets us assess results against a specific rule. Do not present such a small trial as an assessment of all model abilities or a prediction of error frequency in daily work.
A shorter explanation can serve the work better
After checking the first variant, a useful answer is: "Two sets can be prepared. Each consumes two stands and one beam. Four stands exhaust the stock; six beams remain." This is our own explanation of the correct result in the example. It lets an employee check assembly without guessing the meaning of a total of twelve parts.
An approver also needs the inputs and calculation record. A person issuing equipment needs the appropriate items and confirmed actual stock. Tailor explanations to the task and audience. Expanding one text for everyone can hide an important condition among long transitions that do not help anyone prepare the room.
Rules for checking AI work (in Polish) can specify which bases accompany each type of result. A mandatory essay is not the goal. The important thing is being able to check the element on which the next action depends and clearly record what was not established.
Before approval, choose one sentence carrying the decision and ask what could disprove it. In our example the number of stands is enough. If the only answer is another model sentence about its own care, evidence is still missing. Instead of requesting a longer explanation, read the right input and perform the necessary check.







