When an AI gives you a coherent answer, it may feel as if the problem has been understood. Often, something more basic has happened: the model has built the best available story from whatever you put in the prompt.

That distinction matters because your prompt contains only what occurred to you to include. The model may not identify the missing information, and its polished response may prevent you from noticing the gap yourself. This is the central risk of WYSIATI in AI.

What does WYSIATI mean?

WYSIATI means What You See Is All There Is. Daniel Kahneman used the term to describe our tendency to form judgments from the evidence in front of us while failing to account adequately for evidence that is absent.

Consider a question Kahneman uses to illustrate the idea:

Will Mindik be a good leader? She is intelligent and strong.

Most people can feel an answer beginning to form. Mindik sounds promising. Intelligence and strength fit easily into a story about leadership.

But intelligence and strength are not enough to answer the question. We do not know whether Mindik is trustworthy, emotionally steady, decisive, open to criticism, capable of resolving conflict, or inclined to misuse power. The description supplies evidence that supports one conclusion while withholding traits that could overturn it.

Our minds do not naturally leave the space blank. They complete the picture.

Kahneman’s point is not simply that people make decisions with limited information. Every decision involves limits. The deeper problem is that subjective confidence responds strongly to narrative coherence, not just to the quantity or quality of the evidence. A small number of compatible facts can produce a smooth story, and a smooth story can feel complete.

That feeling is dangerous when working with large language models. An LLM also receives a limited description and produces a coherent continuation from it. It cannot inspect the parts of your situation that never entered the conversation.

What did the one-sided evidence experiments reveal?

The strongest direct evidence for WYSIATI comes from experiments in which people judged legal cases after seeing either one side or both sides. The result was not merely predictable bias toward the visible side. Less information did not lower confidence, and often raised it.

Brenner, Koehler and Tversky studied this pattern in a series of experiments published in 1996. In one experiment, 137 students evaluated six legal scenarios. Some received only the plaintiff’s case, some received only the defendant’s case, and juries received both.

Participants then predicted how many members of a 20-person jury would favor the plaintiff:

Evidence presented	Average predicted jurors favoring the plaintiff
Plaintiff’s case only	12.3
Defendant’s case only	9.5
Both sides	11.3

The evidence people saw pulled their predictions toward the side presented. That part is unsurprising. More revealingly, the one-sided groups gave narrower uncertainty ranges than participants who had heard both sides. In other words, exposure to the conflict between the two cases did not merely complicate the decision. It made participants appropriately less certain.

The one-sided participants knew that their evidence was incomplete. They had been told explicitly that they were receiving only one side, yet they did not compensate enough for what was missing. In the authors’ assessment, people failed to compensate sufficiently even when the omission was painfully obvious (Brenner, Koehler and Tversky, 1996).

The qualification matters. One-sided evidence did not always produce lower accuracy, and the defendant-only group performed comparably to the juries on some measures. The responsible claim is narrower: less information did not reliably reduce confidence, and it often increased it.

This is the psychological foundation for understanding what can go wrong with AI-assisted judgment. A visible story commands attention. An absent story does not.

A fluent answer can make missing evidence harder to see precisely because the evidence that remains fits together so well.

How does WYSIATI in AI appear in practice?

WYSIATI in AI appears when a model answers from an incomplete or misleading prompt without first identifying what it would need to know. Recent benchmarks suggest that even strong models often prefer completion over clarification.

The analogy needs care. No study in the research reviewed here tests WYSIATI by name in large language models. WYSIATI is a human cognitive concept, while an LLM is a statistical system trained to produce language. The connection is an interpretive one, based on similar observable patterns rather than identical internal mechanisms.

Those patterns are still striking.

On the IN3 vague-task benchmark, Gemini asked clarifying questions at a rate of 0.118, while GPT did so at a rate of 0.177. On AskMind, both covered only about 12 percent of the missing checkpoints required for a well-grounded response (Zhao, Fang and Cheng, 2026).

The exact numbers should not be treated as permanent properties of these model families. This evidence comes from a recent preprint, and model behavior changes quickly. The broader pattern, however, is clear within the study: strong models commonly attempted an answer instead of resolving the ambiguity first.

False premises create an even sharper test. Under AskBench’s strict two-turn protocol, Gemini answered immediately in 387 of 400 cases containing false premises, producing an accuracy score of 0.010 under the benchmark’s rubric (Zhao, Fang and Cheng, 2026). Instead of challenging the faulty frame, the model usually continued inside it.

Even when researchers reduce the problem to one missing variable, models struggle to identify the right question. QuestBench presents a solvable problem with exactly one piece removed and asks the model to select the clarifying question needed to recover it. Performance was strong on grade-school mathematics but only about 40 to 50 percent on logic and planning tasks (Li, Kim and Wang, 2025).

This is not a simple failure to ask anything. It is a failure to find the consequential omission.

What does an incomplete catering prompt teach us?

An incomplete catering prompt shows how easily a model can convert missing constraints into plausible assumptions. The answer may look useful while resting on decisions the user never knowingly made.

Imagine this prompt:

Plan catering for our company event and keep the costs reasonable.

A model can answer immediately. It might recommend a buffet, estimate portions, suggest several menu categories, and propose ways to reduce waste. The response could be crisp, practical, and completely wrong for the event.

The prompt does not specify:

• How many people will attend
• Whether the event is breakfast, lunch, dinner, or an all-day program
• Where it will take place
• Whether the venue has kitchen or refrigeration facilities
• What reasonable means as a budget
• Which allergies or dietary restrictions must be accommodated
• Whether alcohol is permitted
• Whether guests will be seated or moving between sessions
• Whether service staff, tableware, delivery, and cleanup are included
• How formal the event should feel

Every polished recommendation must either leave these variables unresolved or quietly fill them in.

Suppose the user then adds a leading frame:

Explain why a buffet is the smartest option for our company event.

Now the model has two reasons to produce a confident buffet argument. The prompt requests an explanation rather than an evaluation, and the preferred answer is built into the wording. Research on AI assistant sycophancy suggests that models can lean toward a user’s stated position, while human preference data may reward answers that align with the user’s view even when the aligned answer is less accurate (Sharma et al., 2023).

The resulting answer may be persuasive. It may mention flexibility, variety, and lower staffing requirements. None of that resolves whether a buffet works for this venue, guest list, schedule, accessibility requirement, or budget.

The model is not necessarily discovering that a buffet is best. It may simply be making the visible option sound coherent.

Why can a confident answer be more dangerous than a weak one?

A confident answer can hide uncertainty by turning assumptions into smooth prose. Research also indicates that verbalized confidence in LLM outputs tends to skew too high, although sampling multiple answers and checking their consistency can reduce the problem (Xiong et al., 2023).

One study of mathematical questions with key conditions removed found baseline abstention rates as low as 16.9 and 21.1 percent for two tested models. Yet linear probes of their hidden states detected unanswerability with accuracy above 0.8 (Liu et al., 2026).

That finding suggests a troubling separation. A model’s internal representation may contain a signal that the problem is defective, while its visible behavior still produces an answer.

The practical issue is not whether a model knows in the human sense. The issue is whether uncertainty inside the system reaches the user in a form that can change the decision. Often, it does not.

Fluency intensifies the problem. A messy answer invites scrutiny. A polished answer can close the case too early.

What is the double WYSIATI loop?

The double WYSIATI loop is an argument about how incomplete prompts and coherent outputs can reinforce each other. The user supplies a partial frame, the model completes it, and the resulting fluency makes the original omissions even less visible.

The loop works in four stages:

1. The user compresses reality into a prompt.
   The prompt contains what the user remembers, notices, values, or assumes is relevant. Important constraints may never appear.
2. The model works from that visible frame.
   It may answer rather than ask, fail to locate the crucial missing variable, or accept a false premise.
3. The model produces a coherent story.
   Language models are built to generate plausible continuations. The response can feel deliberate even when it depends on unstated assumptions.
4. The user evaluates the polished result.
   Now human WYSIATI returns. Coherence becomes a cue for confidence, and the response looks like additional evidence rather than an elaboration of the original frame.

This double-loop concept is a synthesis of separate research streams, not an experimentally established finding. The human evidence shows that coherent, one-sided information can inflate confidence. The newer LLM evidence shows that models often answer ambiguous or defective questions, lean toward user framing, and express excessive confidence. Combining those findings yields the warning.

The AI can hand your assumptions back to you with better grammar, stronger structure, and more apparent authority.

That is not independent judgment. It is amplification.

Why does telling the model to be unbiased fall short?

A generic instruction to be unbiased does not identify the missing evidence. Effective interventions make the absent side concrete enough to evaluate.

Human research illustrates the difference. In a fourth experiment, Brenner and colleagues asked participants to rate the strength of the side they had not seen before making their judgment. The average gap between plaintiff-only and defendant-only groups fell from 7.1 to 2.7 (Brenner, Koehler and Tversky, 1996).

The intervention did not eliminate the bias. It cut it by more than half by forcing the missing side into view.

Lord, Lepper and Preston reached a related conclusion in experiments on biased assimilation and hypothesis testing. Asking participants to consider opposite possibilities worked better than merely instructing them to be fair and unbiased (Lord, Lepper and Preston, 1984).

This distinction should shape prompting. Compare these instructions:

• Be objective.
• Identify the strongest evidence that would overturn your recommendation.

The first requests a virtue. The second creates an analytical task.

LLM-side interventions also appear trainable. Rubric-guided training increased false-premise detection coverage in one Qwen model from 0.188 to 0.894, although the model also asked more redundant questions. A separate probe-and-intervene method raised abstention on unanswerable mathematics from 21 to 74 percent for one model while preserving accuracy on answerable questions (Zhao, Fang and Cheng, 2026; Liu et al., 2026).

Better caution therefore has a cost. A model trained or prompted to ask more questions may become less direct on tasks that were already clear. The objective is not maximum hesitation. It is sensitivity to the missing information that could change the answer.

How do premortems and the outside view expose missing information?

Premortems and outside-view forecasting interrupt WYSIATI by introducing evidence the current story tends to exclude. One looks for hidden failure modes; the other compares the case with relevant outcomes beyond the case itself.

A premortem asks the team to assume the plan has already failed and explain why. Gary Klein developed the method to surface risks that enthusiasm and commitment tend to suppress. It draws on prospective hindsight research reported as improving people’s ability to identify reasons for outcomes by 30 percent (Klein, 2007; Mitchell, Russo and Pennington, 1989).

The wording matters. Asking whether a plan might fail leaves success as the default story. Assuming failure temporarily changes the frame, making neglected risks easier to retrieve.

The outside view performs a different correction. Instead of concentrating on the details that make a case feel unique, it asks what happened in comparable cases.

Lovallo and Kahneman describe a curriculum-development team that estimated completion would take 18 to 30 months. When a team member considered comparable projects, the picture changed dramatically: about 40 percent had failed, and those that finished took seven to ten years. The project eventually took eight years (Lovallo and Kahneman, 2003).

Reference class forecasting turns this correction into a practical sequence:

1. Select a genuinely comparable class of cases.
2. Find the distribution of outcomes within that class.
3. Place the current case within that distribution, adjusting only for supported differences.

For AI use, this means refusing to let the model remain inside your case narrative. Ask for base rates, comparison groups, and sources. If reliable base-rate evidence is unavailable, the model should say so rather than manufacture precision.

How strong is the evidence for WYSIATI in AI?

The human evidence is older, peer-reviewed, and comparatively stable. The LLM evidence is newer, much of it comes from 2025 and 2026 preprints, and the findings may shift as models, training methods, and interfaces change.

That difference does not make the recent evidence useless. It changes how confidently it should be interpreted.

Several strands point in a similar direction:

• Models often answer underspecified prompts instead of clarifying them.
• They struggle to identify the decisive missing variable in logic and planning tasks.
• They may accept false premises rather than challenge them.
• They can lean toward the user’s expressed view.
• Their verbalized confidence tends to run high.
• Cognitive-bias testing across 20 LLMs found evidence of all 30 tested biases in at least some models, including framing, with no clear relationship between model capability and lower bias (Malberg et al., 2024).

Still, these studies do not establish that an LLM has Kahneman’s System 1, nor that all models behave alike. They support a practical comparison: both people and current AI systems can produce confident judgments from an incomplete visible frame.

The safest conclusion is operational. Do not assume that a stronger model will notice the gap you missed. Design the interaction so the gap has to be examined.

Which five prompts can reduce WYSIATI in AI?

The most useful prompts make missing information visible before requesting a recommendation. The following five patterns translate established countermeasures into practical LLM use, although they have not been tested together as a single intervention.

1. How can you make the model identify missing information first?

Ask the model to diagnose the prompt before answering it. This creates a separate stage for finding omissions rather than allowing assumptions to disappear inside the final response.

Prompt pattern:

Before answering, list the information you would need to give a good answer. Identify which missing details could materially change your recommendation, then ask me for them.

The phrase materially change is important. Without it, the model may generate a long intake form full of low-value questions. You want decision-relevant uncertainty, not bureaucratic completeness.

For the catering example, headcount, dietary restrictions, venue facilities, format, and budget could change the recommendation. The color of the event signage probably could not.

2. How can you force the unseen side into view?

Ask for the strongest case against the preferred option. This is more useful than requesting balance because it gives the model a concrete adversarial task.

Prompt pattern:

Make the strongest evidence-based case against this option. Identify the assumptions that must be true for it to succeed and the evidence that would overturn the recommendation.

This pattern applies the logic behind rating the unseen side and considering the opposite. It does not guarantee neutrality, but it makes the counterstory visible enough to judge.

3. How can you use a premortem with an LLM?

Assume the plan has failed, set a realistic time horizon, and ask for causal explanations. The exercise is designed to retrieve risks that the success narrative leaves out.

Prompt pattern:

Assume this plan failed 12 months from now. What were the most likely reasons? Rank them by impact and likelihood, then name the earliest warning sign for each.

Adding warning signs turns the output from imaginative pessimism into something that can be monitored. The result remains a hypothesis, not a forecast, but it gives the decision-maker a better risk map.

4. How can you make the model take the outside view?

Ask what usually happens in comparable cases, where the base rates come from, and whether the comparison is genuinely relevant. This moves the analysis beyond the distinctive details of your own story.

Prompt pattern:

What usually happens to organizations like mine when they make this decision? Identify an appropriate reference class, provide relevant base rates, cite where they come from, and explain whether my case differs enough to justify an adjustment.

Do not accept invented base rates. If the model cannot provide a credible source, treat the claim as unverified and seek the evidence elsewhere.

5. How can you remove a leading frame from the question?

Ask for an evaluation before asking for a justification. A neutral comparison gives the model permission to reject the option rather than merely decorate it with reasons.

Instead of:

Why is offering a discount a smart idea?

Ask:

Should I offer a discount? Compare it with the strongest alternatives, state what information is missing, and explain what conditions would make each option preferable.

The same correction applies to hiring, pricing, product launches, investments, strategy, and policy. If the conclusion is already embedded in the question, the model may optimize for advocacy rather than judgment.

What should change about the way we use AI?

The biggest improvement is not a more elaborate prompt. It is a change in sequence: expose uncertainty first, then ask for an answer.

Most people use an LLM in a single pass. They describe the situation, request a recommendation, and evaluate the result. That workflow gives coherence too much power.

A safer workflow separates diagnosis from judgment:

1. Ask what is missing.
2. Resolve the most consequential gaps.
3. Ask for the strongest opposing case.
4. Run a premortem.
5. Check the outside view.
6. Request a recommendation with explicit assumptions and uncertainty.

This takes longer. Sometimes it will be unnecessary. For low-stakes, reversible tasks, a direct answer is often exactly what you want.

But when the decision is expensive, consequential, or difficult to reverse, speed is not the only objective. The unanswered question is part of the answer.

WYSIATI in AI is not merely a problem inside the model. It begins in the prompt, continues through the model’s response, and ends in the confidence we place in a story that sounds complete. The most important prompt may therefore be the one that interrupts the answer before it arrives:

What would we need to know to discover that this story is wrong?