We are eighty percent confident: does the team know what that means?
Record team forecasts so you can revisit outcomes. Separate probability from goals and examine the limits of confidence, comparisons and updates.
Article language: English

In this article
"We have an eighty percent chance of finishing on time" can mean several things: we have strong evidence, we really want to finish, or it seemed time to provide a number. The listener often cannot tell which meaning they have received. Only by recording a specific prediction and later returning to its outcome can a team examine how it uses the language of confidence.
Forecast calibration concerns the relationship between stated probabilities and the frequency of events in an appropriate group of predictions. A single outcome cannot establish it. If a highly likely event does not occur, that alone does not prove that the stated chance was meaningless. Look at many clearly resolved questions and preserve the conditions under which they were assessed.
Imagine a company preparing a digital catalogue of materials. The team wants to predict more accurately whether it will receive a complete set of data before a scheduled review. It also uses an AI assistant to organise information about cases. It does not expect prophecy. It wants to see when it speaks too confidently, what it leaves out and what changes its assessment.
Start with an event that can later be resolved
"The project will go well" has no sufficiently clear endpoint. In our example the question could be: "Will we receive the complete set of materials described in this list by the agreed deadline?" Specify the date, what completeness means and where the result will be confirmed. If several people interpret the question differently, their numbers do not describe the same prediction.
Separate the forecast from the goal. The company's goal may be to receive the material before Friday's review. The forecast expresses its assessment of the chance that this will happen given current knowledge. The plan identifies actions the team will take. The three statements can be recorded side by side. None needs to pretend to be another just because the calendar has one date field.
Strategic scenarios (in Polish) answer a different question: which alternative conditions should we consider? Calibration concerns the later comparison of numerical assessments with resolved events. Both ways of thinking can be useful. Do not, however, assign an arbitrary probability to a scenario simply to put it in a forecast table.
Start with a type of event for which the company can retain both the data and the outcome. In our catalogue example, material completeness is described before the assessment. Adding a newly required attachment after the deadline changes the question. Show that change rather than retroactively treating the earlier material as incomplete under the new definition.
What forecasting research contributed
In a 2014 publication, Mellers and co-authors studied forecasts of geopolitical events in a tournament held from 2011 to 2013. They distinguished calibration from the ability to distinguish more likely events from less likely ones. Under the study's conditions they found benefits from specified probability training and team collaboration. This does not guarantee accurate predictions of a Polish company's deadlines.
IARPA's description of the ACE programme discusses eliciting and combining probability assessments and empirically testing them against real events. It describes the aim of a research programme. For our example, the relevant point is connecting a prediction to its later resolution. Collecting opinions at a meeting does not by itself complete such a test.
A company can start by retaining the question, assessment, its rationale and the outcome. Such a record makes it possible to see whether the team's shared language of confidence is grounded in experience. We do not need an immediate title for forecaster of the month; we need a way to learn without rewriting memories.
In your own decision note for management, separate the predicted outcome from the choice of action. The guide to a management decision note (in Polish) explains how to prepare such a document. A decision can be reasonable even if an unlikely event ultimately occurs. Assessing the choice requires knowing what was available at the time, not merely the later story of success or difficulty.
Similar events help explain a number
In our catalogue example, the team can return to earlier requests for data. First it checks which were genuinely comparable: the same kind of material, a similar scope and a comparable stage. It does not add all customer contacts together, including a quick address confirmation and preparation of an entirely new specification. The common label "waiting for a reply" does not ensure a similar course of events.
Suppose solely for illustration that eight of ten documented earlier cases finished before their specified deadlines. This is information about that small set. It does not guarantee the same frequency in future. The team should check whether the current request differs in scope, the recipient's availability or the completeness of the input provided. The number organises the question; it does not remove the need to read the case.
When comparable data is missing, show that gap alongside the assessment. Do not add apparent precision merely because a form accepts a percentage to two decimal places. A preliminary judgment can be recorded with a clear basis and a condition for revisiting it. The team should know whether it is drawing on repeated experience, one signal or its own assumption.
The agreement of a customer advisory board (in Polish) is not automatically the frequency of future purchases across an entire market either. Participants may explain their own needs well, but their selection and the scope of questions set the limits of the conclusion. If the team wants to record a forecast of a specific event, it must separately define its basis and how it will later be resolved.
An update should not erase the earlier assessment
In the fictional example, the initial forecast is made before some data arrives. The next day the customer sends materials but indicates that one attachment is missing. This is new information. The team can change its assessment while retaining the earlier record, the time of the change and its rationale. This shows how it understood the situation before and after the message.

Do not overwrite the first number with the last one just before the deadline. A later forecast may use much richer knowledge. When comparing assessments, identify when each was made. An assessment made after completion has been confirmed is no longer a prediction of the same event under the earlier uncertainty.
Record the company's action separately. If the team sends a follow-up question after making a forecast, it may change how the case develops. The outcome can still be recorded, but the intervention must be retained when interpreting it. Forecasting in an organisation is not always done from the sidelines. Sometimes the person asking the question later helps bring about the result.
A limited business experiment (in Polish) can test a particular change. Calibration of assessments is a separate task. Do not attribute improved outcomes to forecast accuracy alone if the contact method, scope of material and deadline were changed at the same time. Preserving the sequence of events helps avoid that shortcut.
One customer reply may resolve a different question
Return to the catalogue. Suppose the list requires a product description and two specified attachments. The customer replies before the deadline, attaching the description and one file. The assistant marks in its summary that a reply arrived. The team should check which event has actually been confirmed. The question of whether a message arrived has a different outcome from the question of whether the originally specified set is complete. In this fictional workflow, distinguish the message status from the status of the expected material.
If completeness was recorded in the forecast table, the presence of a message alone must not be accepted as a positive outcome. The responsible person compares what arrived with the agreed list. If the missing attachment arrives later, they retain both times. This distinguishes the reply time, the time the complete set arrived and the time its assessment ended. Otherwise the result looks better only because the meaning of "received" changed along the way.
Now imagine the company agrees a different deadline after the first message. The old question should be resolved against the original date if that can still be determined. The new date can be the basis for the next forecast. There is no need to pretend the earlier case never existed. The team can consider the change of plan sensible while retaining the fact that the earlier forecast deadline was not met.
After such a case, an improvement may concern the inputs used for assessment. Instead of a single "customer replied" status, the office needs to distinguish whether the specified complete set arrived. This does not yet prove that later forecasts will be more accurate. It does provide a better basis for forming and checking them. In later assessments, the team records that it is now using new information so it does not mix it with the earlier way of describing cases.
That explanation can be more useful than debating whether the number should have been seventy or eighty percent. If two people predicted different events, changing the number alone will not repair the comparison. First agree on the question and the evidence of the outcome; then consider how well founded the stated confidence was.
Returning to outcomes requires the whole set
After the agreed time, resolve the questions under the accepted rules. Mark those that could not be assessed and explain why. Do not discard an inconvenient case merely because it lowers the apparent accuracy of the set. If the event's definition changed, retain that information. If the outcome is known, retain its confirming source instead of relying on participants' memories.
Compare stated chances with the later frequency of events in appropriate groups of assessments, for example those close to eighty percent. Show the number in each group as well. A few high-confidence forecasts do not provide a strong basis for judging an entire forecasting approach. The kind of questions matters too. Easy, almost settled events and difficult cases with many unknowns can yield different results. An overall percentage of correct outcomes is hard to use without that information.
Do not confuse cautious language with usefulness. A team that always gives the same intermediate chance may avoid confident errors but distinguish very little. In the Mellers study, calibration and discrimination between events were separate parts of the assessment. A company's own table should therefore help show whether new information actually changes judgment, not merely whether people choose their words more cautiously.
When an outcome surprises you, you can reconstruct the causes of the problem from evidence (in Polish). The questions are: what did we miss, what happened later and which assumption was wrong? An error alone does not justify assigning blame or declaring a single cause for all similar cases.
Where does AI fit?
If an assistant organises information about cases, separate its record from the team's judgment. It should be clear which data was used and what was missing. A number generated in an answer does not acquire calibration by its appearance. To evaluate a tool's forecasts, retain its specific predictions and later outcomes, within the appropriate limits of use.
At Aeneia Audit, a conversation can start with the information flow on which the assessment of an AI use depends. Prepare a specific question, input data and a way to check the outcome. Separating observation, hypothesis and prediction makes it easier to assess what the tool actually does and where human judgment is still needed.
Aeneia Audit
Present the data and the expected result of using AI for a specific task. Ask whether the scope can include discussing sources, limits of the conclusion and ways to check the result after the event. Organise the basis for evaluating an AI use
A well-preserved forecast does not need to end with an impressive "I told you so". It can end with a more honest sentence: "This is what we knew then, that information arrived later, and now we are changing how we assess the situation." For the next decision, that record is more valuable than a shared memory in which everyone sensed the right result from the start.








