Skip to content
Interface language: English
Audit / Signal

An AI confidence percentage needs its own verification

Check where an AI confidence score comes from, what counts as correct and how groups of results compare.

Article language: English

Illustration: part of a stone column, with a thin line marking a crack.
In this article

A field saying "confidence: 93 percent" can look like a measurement even when the model supplied the number at a user's request. Before it directs cases to a person, establish its meaning. Is it meant to predict whether this answer is correct, measure similarity to a retrieved document, or express a score someone labelled as a percentage without justification?

Each number may serve a task, but its role must not change on the way to the interface. A displayed probability of correctness needs evidence that its declarations correspond to outcomes in this use. Fluent prose and a precise-looking figure do not perform that check.

Establish what the percentage measures

In a fictional process, an assistant answers questions about workshop-room equipment. Each answer has a number. The team wants to read the cases with the lowest stated confidence first. That may be sensible, but only if the number actually helps find errors. The field's existence does not establish that.

Ask the supplier where the indicator comes from. The model might write a number itself, a separate mechanism might calculate one, or the system might use another measure, such as a retrieval score. Record the method, version and exact meaning. A document's match to a question does not become the probability that the answer is correct because someone changes its label.

Likewise, do not equate the probability of generating a piece of text with the probability that the whole statement is true. A model can easily predict a common expression that does not fit a company's current instructions. The reader needs to know what was measured before interpreting the result. A percentage needs a definition beyond being between zero and one hundred.

The 2023 study by Tian and colleagues compares ways of eliciting confidence, including model statements and model probabilities. It concerns particular models and question sets. It shows that the method matters; it does not establish that any present-day numerical declaration is calibrated for a company's process.

Define the object of the claim. "I am confident in the answer" is unclear when the answer contains five facts and a recommendation. Must all facts be correct? Is the right dimension enough? Is agreement with a document sufficient when that document itself is outdated? Without a shared rule, two assessors may label the same output differently.

In our example, choose a narrow task: answering an unambiguous question about the number of seats in a room according to approved instructions. Record permissible answers and conditions for stating that the evidence is insufficient. This does not validate every later recommendation. It provides a task whose outcome can be established for an initial check of the number.

Suppose the instructions for fictional room B allow ten workshop places, while an equipment list mentions twelve chairs. The question asks for places according to the instructions. "Twelve" uses a real number from another field but fails the criterion. A declaration of 93 percent settles nothing. The assessor needs the relevant passage and the meaning of the question.

If one sentence states the ten-place limit and an incorrect booking condition, decide beforehand how to score it. You may separate claims or require the entire answer to be correct, depending on the use. Do not change that rule after seeing the percentage. Calibration against one criterion does not establish calibration against a broader one.

Include a case where the instructions do not contain the required limit. The evaluation then concerns whether the assistant recognises the lack of evidence, not whether it guesses a number known to the assessor from another document. This prevents punishing a correct response for unavailable knowledge or rewarding a guess. All room examples are for designing acceptance, not real model assessments.

Calibration concerns groups of results

Calibration describes the relationship between declared probabilities and the observed frequency of the relevant outcome in groups. One answer marked 90 percent can be wrong even with a well-calibrated mechanism. One mistake does not settle the evaluation, just as one correct answer does not establish that the numbers describe the rest well.

The scikit-learn documentation describes calibration curves comparing average predicted probability with observed outcome frequency in groups, and shows group counts. It concerns classifiers. Here we apply a similar comparison between declared answer correctness and later assessment under a defined criterion.

For this example, a new fictional set of forty records was constructed. Each has an assigned declaration and correctness label. The values were deliberately chosen to show an error concealed by comparing averages. They are not model answers or results from a product test. This is not the recovered historical dataset.

Ten records declare 90 percent, of which six are labelled correct: an observed 60 percent. Twenty declare 70 percent, with fourteen correct: 70 percent. The remaining ten declare 50 percent, but eight are correct: 80 percent. These are illustrative calculations, not results from an AI service.

Across all forty, twenty-eight are correct, giving 70 percent. The weighted mean of the forty declarations is also 70 percent. That mean comes from the assigned confidence values and group sizes, not the correctness labels. The aggregate figures agree, yet the highest declaration does not match its group's results and the lowest understates its group's frequency.

Aeneia conceptual photography

That disproves the inference "mean confidence equals accuracy, so everything is calibrated". It does not estimate the quality of a real process. The small constructed groups are not a sample of users. A real assessment needs cases resolved independently and uncertainty displayed, especially for small groups.

Calibration of a team's forecasts (in Polish) also returns to predictions and their outcomes. Here the source of the number and its link to a particular system answer must additionally be established. A cautious human tone is no substitute for that information; nor is a model's numerical declaration.

Keep the timing and method of assessment

A declaration before solving a question can concern the chance of completing the task. A score after receiving an answer can concern that particular response. Do not combine them in one series without explanation. The later number has a different information state, especially if the system read a document or received a tool result in between.

CALIBER, published in October 2026, distinguishes these two moments and their assessment targets. Its authors study alignment between declarations before and after reasoning and different training targets. Their method and results do not validate our process. For acceptance, establish which quantity the indicator actually displays.

Alongside the answer, save the instruction requesting the score, sources, mechanism version and time when the number was assigned. If a corrected source leads the model to another answer, that is a new result to assess. Source freshness in AI research (in Polish) is a separate check; high confidence does not update an old instruction. Do not attach an old score to a revised answer merely because both remain under one question in chat history.

If you want an assessment label as independent as possible, do not show the human evaluator the confidence score initially. First compare the answer with the criterion and source. Then compare that label with the declaration. Otherwise "93 percent" may become a suggestion the assessor unconsciously repeats.

In an AI pilot (in Polish), retain errors and unresolved cases. Do not automatically call an unassessable answer correct or silently delete it. Show how many answers could be checked and why the remainder were excluded. Otherwise a favourable result may describe only the easiest part of the work.

Record a missing number separately from a declared zero. Do not insert a default just to fit a table. An out-of-range value also needs explanation before calculation. Report how many cases were excluded from assessment of the number and why. They remain cases in the system's work even if they cannot be placed on a calibration curve.

A number can order a queue and still misstate the chance

Separating better outcomes from worse ones and matching percentages to frequencies are different needs. A mechanism may help identify errors while its displayed values need correction. It may also assign everyone a similar number close to average accuracy and do little to select cases for review. Assess the use the company actually plans.

In the constructed example, the group with the highest declaration has fewer correct outcomes than the lowest group. A rule to skip high percentages would therefore let some problems through. That observation concerns constructed data. In real acceptance, compare the proposed ordering with independently identified errors and check a sample of cases otherwise left unattended.

A suitable threshold does not follow from a universal 80, 90 or 95. It depends on consequences, task and evidence for the indicator. Preliminary sorting of working notes may use different rules from sending answers automatically. If there is no basis for an automated decision, the percentage itself does not justify enabling it.

With Aeneia Audit, you can discuss how an AI indicator is used in the next step: who reads selected answers, how errors return to the process and what remains under human control. Agree any review scope separately.

Keep a separate check after the fix

If the instruction eliciting the score changes, do not compare new numbers with old ones without recording that change. A different method, model or source can alter the indicator's meaning. The set used to improve the mechanism should not be the only basis for evaluating it afterwards. Keep a separate sample whose result has not already been used to tune the rule.

Do not fix the number merely by adding cautious words. "Probably" and a field saying "99 percent" may send conflicting signals. An AI explanation (in Polish) can help explain an output but cannot by itself prove calibration. That must be assessed on outcomes with a defined method.

When there is no basis for an answer, the system still needs an appropriate response to uncertainty and an available next step. A low number without identifying the missing source may leave the user with the same problem. The indicator should help decide what happens next, not merely warn that answers can be wrong. See the article on when AI should say it does not know (in Polish) for a related perspective.

The fictional assistant's owner should ultimately be able to say what the number means, which answers it was checked against and what action it may support. Until those elements exist, 93 percent remains a declaration to investigate. Decimal places do not take the place of evidence.

Aeneia / Aeneia editorial

Editorial article. Source links are included in the text.

About Aeneia

Brands we have worked with

  • L’Oréal
  • Garnier
  • Netflix
  • Carrefour
  • Warner Bros. Discovery
  • Biedronka
  • HBO Max
  • Wolt
  • Żabka
  • COSRX
  • Embassy of China in Poland
  • Mixa
Aeneia / Initium

From knowledge to a decision

Tell us what you want to put in order. We will agree on where to start.

Contact Aeneia

We partner with and use solutions from:

  • Google
  • OpenAI
  • Microsoft
  • Stripe
  • Anthropic
  • Linear