When AI should say it does not know: check a good refusal too
When is an AI refusal correct and when does it hold up work? Check missing sources, unclear questions, unnecessary handoffs and real routes to help.
Article language: English

In this article
"I do not have enough information" can be an assistant's best answer. It can also be a polished way of avoiding a simple task. A company needs to distinguish these situations before treating the number of refusals as evidence of quality. A tool that stays silent after every question does not invent facts. It would still be hard to recommend it on that basis.
A good refusal identifies the missing basis, preserves what is known and leads to a feasible next step. It need not end the conversation. Sometimes the assistant only needs to ask about a product variant, sometimes it must find the right document, and sometimes it should hand the case to someone with the right authority. AI evaluation should cover that choice too.
Three reasons for silence that sounds similar
Imagine a fictional studio assistant handling orders for desk organisers. The tool uses an approved catalogue and enquiry-handling instructions. A customer asks for a model's width without naming the model. The information exists, but the question does not specify the item. Asking for clarification is a sensible response.
A second customer names the model and asks when it will be available. The catalogue has no stock information, and the assistant cannot access the order system. Clarifying the name will not fix that. Another source or a person who can confirm the date is needed. More questions to the customer would only prolong the route to the same missing information.
A third message asks for an assessment of the load capacity of a shelf the studio neither makes nor describes. The matter falls outside the available material and the assistant's task. It should say what it can explain and what it cannot confirm. An honest boundary is more useful than generic advice that sounds expert because it contains three technical nouns.
These answers may begin similarly but require different next actions. Give the team a recognisable distinction between an unclear question, a missing source and a matter outside scope. Record an access failure separately. A failed document retrieval does not prove the document does not exist. Once access is restored, the same case can be checked again while keeping the earlier result as an observation of the failure.
In RAG-based workflows (in Polish), check where the basis was lost. Retrieval may have missed the right material even though it is in the approved collection. Calling every such case a good refusal hides a retrieval weakness. An appropriately limited answer can coexist with a fault earlier in the process.
Allow "I do not know", then test it
Claude's documentation on reducing hallucinations recommends explicitly allowing uncertainty and checking claims against source material. Such an instruction can help reduce unsupported answers. Its presence in a prompt is not, by itself, a test result for your process.
In a working case, record how the assistant should handle the missing organiser name. Then send a message with the name and the relevant catalogue passage. The tool should now be able to answer. If it refuses in both situations, the team may have taught it a cautious tone rather than recognition of available information.
Prepare a pair of questions differing in one important condition. In the first, the source gives external dimensions and the recipient asks for those dimensions. In the second, the recipient wants the size of the internal space. Check that the model does not transfer a number between these roles. Similar words do not create a missing measurement.
Do not require the assistant to declare itself absolutely certain. That declaration is still text in an answer. If you record a confidence score, define what the number means. Then compare it with correctness on tasks the tool actually performs. "I am confident" is easy to write. It does not give the studio the missing specification.
Two Axes of LLM Abstention, published in July 2026, separates answer correctness from recognising whether a question can be answered at all. Its author investigates these axes on a defined set of models and tasks. This is a reason to test the two aspects separately, not a ready-made result for every company assistant.
When evaluating an answer, identify its basis. Was the selected dimension actually in the material? Did the question require information that was missing? Could a partial answer be given without implying the rest? Your own set of decisions will be more useful than observing that the model sometimes sounds bold and sometimes restrained.
A question may smuggle in an unconfirmed fact
A customer writes: "Why did you discontinue the version with the wooden divider?" The available catalogue shows various products but contains no information about such a withdrawal. The assistant should recognise that the question itself does not confirm the event. Inventing a reason would rely on the questioner's assumption rather than the studio's knowledge.
It can say it has no confirmation that the version was discontinued and ask for the product name or a link. If it finds a current product page, it can describe what that page says within its limits. It should not declare that the customer invented the event. The assistant's materials may also be incomplete. Its job is to separate a confirmed fact from a question requiring investigation.
Prepare a counterpart with an actual, documented withdrawal. The expected answer will then differ. This pair shows whether the tool uses its basis or automatically challenges every question containing "why". Indiscriminate suspicion can also make service harder. The right result depends on the information available in that test.

Also be careful with "that is not possible". The absence of a description in a catalogue may justify only "I cannot confirm this from the information available". The first closes the case; the second states the boundary of knowledge and allows further checking. Evaluate this difference in meaning even if both messages sound equally polite.
Refusal can have a cost
Excessive withholding sends ordinary cases to people. The team starts copying dimensions that were available from the start. The user waits even though the system has the necessary material. To evaluate usefulness, record these cases alongside unsupported answers. Both kinds of error change the workflow.
Do not combine them into one number without showing its components. Nine correct refusals and one unsupported answer may mean something different from one correct refusal and nine unnecessary handoffs. You need to know which tasks produced the errors and what happened next. These example numbers illustrate a comparison; they are not research results.
Set acceptance criteria before seeing the answers. If the source contains the organiser's width and the question unambiguously identifies the model, an unsupported refusal should be recognised. If the question asks for a date without stock data, a smoothly presented date should not earn a point for politeness. Good style has a role, but it does not run the warehouse.
Before discussing results together, ask two people to assess selected cases against the same criterion. If one sees the absence of stock data as a reason to ask about colour and the other as a reason to hand the case over, clarify the disagreement in the evaluation instructions. Voting alone will not establish the right path. You need a decision from someone who knows the actual order-handling process.
The reference standard should cover acceptable meanings, not one compulsory sentence. "Give the organiser's name" and "Which model do you mean?" can achieve the same clarification. Keep a justification based on the material and task for each decision. A new team member can then understand the assessment instead of merely copying the author's favourite phrase.
Also check the answer after the data is supplied. If the customer has given the name, the tool should use it and proceed. Asking for the same detail again reveals a conversation continuity fault. A one-off test of a correct clarification could miss it, even though the user sees it in the second message. Acceptance therefore needs both the right refusal and a sensible return to the task.
Keep this distinction in automated evaluation too. An AI model acting as a judge (in Polish) may reward a smooth, complete answer even when completeness comes from an invented detail. Give it an explicit criterion and compare its decisions with human assessment. Then inspect disagreements.
Record whether a refusal concerns the whole question. The assistant may know the material and dimensions but not the delivery date. If those parts can be separated safely, it can provide the available facts and state what is missing. Complete silence is not the only way to respect boundaries.
The next step must really be available
"Contact support" is of little help if the recipient is already speaking to support through the same form. The next route should reflect what the company can do. It may be a handoff with the description already provided, an active contact channel or a clear request for one missing detail.
In the fictional studio, the assistant passes the date question to a person with access to orders. It keeps the product name and the customer's question. It does not add that a date has been reserved, because that did not happen. The recipient should understand the difference between receiving a question for checking and confirming availability.
If the handoff requires a tool action, check its result too. When another agent performs the next step, also verify the task handoff (in Polish): it should receive the question, available material and the identified gap. A generated sentence saying "I handed the case over" does not prove a ticket was created. An observable process state is needed. If the operation fails, the answer should reflect the failure and give an available alternative. Otherwise the model refuses to provide information and then invents successful help.
For an automation exception queue (in Polish), establish responsibility for the case. A record on a list does not mean someone will pick it up. The person taking over needs the question, the identified gap and the material the assistant actually had. Starting the conversation again from scratch undermines the purpose of the handoff.
During acceptance, follow the whole path from the user's perspective. Read the refusal, take the indicated step and see what happens next. Check whether the question returns to the same assistant without a new source. Politeness in a loop is still a loop, even if it wishes you a good day each time.
Keep cases that spoil a comfortable score
After each fix, retain an example of an unsupported answer and one of an unnecessary refusal. Do not select only the group where the new instruction looks good. Prepare fresh situations too. Independence of a test set (in Polish) matters when assessing the ability to withhold an answer as well.
In training a team to verify AI (in Polish), show these pairs side by side. A staff member should know when no answer is correct behaviour and what basis to look for before accepting it. They should not have to judge quality by the model's tone or apologise to a customer for a boundary the company agreed to respect.
For a discussion with Aeneia Audit, bring the question, the available source and the answer that made the team hesitate. We will establish whether what was missing was information, access, clarity or the right decision. The fix depends on that distinction. Adding "be more accurate" to a prompt will not solve all four.










