proof42.com / Articles / Explainer

Why AI Makes Things Up: It Fills In What It Half Knows

Fabrication stems from statistical pattern completion, not total ignorance, shifting where you must look for errors.

By Zoltán Varga2026-11-054 min read

Completion, not lookup

Large language models do not function as databases. They do not retrieve facts from a static list when you ask a question. Instead, they predict the next word in a sequence based on statistical probabilities found in their training data. This mechanism creates a continuous flow of text that mimics human conversation but lacks a direct link to ground truth.

When the model encounters a gap in its knowledge, it does not stop and admit ignorance. It calculates the most likely continuation based on the patterns it has seen before. This behavior turns the system into a completion engine. It fills missing pieces with words that fit the surrounding context, regardless of whether those words are factually correct.

You must understand this distinction to trust the output. A lookup system returns a specific record or says it cannot find one. A completion system always provides an answer because its primary function is to generate text. The answer appears real because it follows grammatical rules and logical flow, even if the underlying facts are invented.

Half-knowledge is the danger zone

The greatest risk of fabrication occurs when the model possesses partial information. It does not invent facts about things it knows nothing about. It invents facts about things it knows partially. When the data is incomplete, the model attempts to bridge the gap using similar patterns from its training set.

This phenomenon creates a specific zone of high error rates. The model recognizes the topic enough to sound authoritative but lacks the specific details to be accurate. It combines fragments of real information with plausible-sounding fiction to create a seamless narrative. The result looks convincing because it is built on real concepts, just arranged incorrectly.

Familiarity increases this risk. Well-known entities often trigger higher rates of fabricated citations. The model expects a source to exist for famous subjects. If no verified source is immediately available in its context window, it generates a fake reference to satisfy the expectation of completeness. This creates a paradox where fame makes the output less reliable.

Confabulation explained

Confabulation describes the unconscious creation of false memories to fill cognitive gaps. In human psychology, this happens when a person constructs a narrative to explain a missing piece of memory without realizing the story is false. The individual defends the claim with confidence, mixing real details with invented ones to maintain coherence.

Artificial intelligence exhibits a similar behavior. The system interpolates from training data to produce confident but incorrect outputs when information is incomplete. It does not intend to deceive. The process is automatic and driven by the mathematical need to complete the sequence. The model treats the generated text as valid because it fits the statistical profile of the request.

This differs from lying. Lying requires intent to mislead. Confabulation occurs without awareness of falsehood. The model believes the output is correct because it is the most probable continuation of the input. You cannot reason with the model to stop this behavior because the mechanism is built into its architecture, not a choice it makes.

Why fluent answers feel true

Fluency acts as a powerful signal of truth for human readers. When text flows smoothly and uses correct grammar, you instinctively assume the content is accurate. This bias makes it difficult to spot errors in AI-generated responses. The model optimizes for this fluency above all else, creating a veneer of reliability.

Ethan Mollick notes in his book Co-Intelligence that the model often prioritizes making the user happy over being accurate. If you insist on an answer about something it does not know, it will fabricate one to fulfill the request. The goal of providing a satisfying response overrides the goal of factual precision. This dynamic drives the production of plausible falsehoods.

The confidence of the output masks the absence of grounding. The model speaks with certainty even when it is guessing. This creates a tension where the syntax suggests truth while the semantics may be entirely fictional. You must ignore the tone and focus strictly on the evidence provided to avoid being misled by the style.

What this means for your checks

You must treat every AI output as a draft that requires external validation. Do not accept citations, dates, or names without independent verification. The model generates these elements to complete the pattern, not because they exist in reality. Check the source document directly to confirm the claim matches the text.

Look for signs of half-knowledge in the response. If the answer covers a broad topic with specific but unverifiable details, it may be confabulating. Ask the model to provide the exact URL or page number for any claim. If it cannot provide a working link or a verifiable document, treat the information as suspect.

Adjust your prompts to reduce the pressure to fabricate. Explicitly tell the model to state when it does not know an answer. Request that it only use information present in the provided context. This limits the scope of its statistical completion and forces it to rely on grounded data rather than internal patterns.

AI-assisted: drafted with a language model from the author's research notes, then edited and checked before publication. Author: Zoltán Varga, AI visibility researcher and AI ecosystem architect, Budapest.