proof42.com / Articles / Analysis

Ghost Citations: Why Real Links Don't Say What AI Claims

A pre-registered study reveals that checking if a link works misses the majority of AI hallucinations because the page exists but supports the wrong story.

By Zoltán Varga2026-11-057 min read

Dead Links Versus Ghost Citations

Most users assume an AI hallucination means a broken link or a non-existent page. This assumption creates a dangerous blind spot in verification workflows. A link can return a 200 status code and load perfectly while containing entirely fabricated information about the subject at hand. This phenomenon is known as a ghost citation. The URL is valid, the domain is legitimate, yet the text on the page does not support the specific claim made by the AI.

In a pre-registered study involving 700 observations across two major AI platforms, researchers distinguished between two types of failure. The first type involves HTTP failures where the link simply does not resolve. The second type involves content failures where the link resolves but the material is irrelevant. The data showed that relying solely on HTTP checks captures only a fraction of the total error rate. The average HTTP failure rate was 7.5%, while the content failure rate among checked links was 50.5%.

This discrepancy means that standard verification tools miss six times more errors than they detect. When a user clicks a link and sees a webpage, they often assume the AI was correct. The page exists, so the logic follows that the citation is valid. This logic fails when the page describes a different company, a different law, or a different medical study. The visual confirmation of a live page masks the underlying fabrication of the connection between the claim and the source.

The distinction matters because dead links are obvious errors. Ghost citations are subtle errors that reinforce confidence in incorrect information. A user sees a professional website with a valid URL and accepts the AI's summary as fact. The verification process stops at the link check, leaving the content mismatch undetected. This structural flaw allows misinformation to persist even when the infrastructure of the web remains intact and functional.

Measuring Link Checks and Content Checks

To measure the true extent of fabrication, researchers implemented a two-step verification protocol. The first step checked the HTTP status of every generated URL. The second step retrieved the page content and compared it against the entity mentioned in the AI's response. This approach required a retrospective analysis of 110 observations where content could be successfully fetched. The results highlighted a massive gap between link availability and content accuracy.

The study found that 98.6% of the sampled URLs contained fabricated content relative to the specific entity requested. In the context of the experiment, which used fictional companies to ensure a zero-tolerance baseline for truth, any returned link was inherently a hallucination. However, in real-world scenarios, the error manifests as a substitution. The AI provides a link to a real company that sounds similar to the target but is not the target. The content check identifies this substitution where the link check sees only success.

The ratio of content errors to HTTP errors was 6.7 to 1. This metric demonstrates that the majority of source failures are hidden behind working URLs. If a verification workflow only pings the URL, it misses nearly 85% of the actual errors occurring in the output. The data suggests that the AI model is confident enough to generate a valid domain name but lacks the specific knowledge to match the content to the query. The result is a high-confidence error that looks like a low-risk success.

Platform differences emerged in how these errors were distributed. One platform generated more dead links, while the other generated more ghost citations. Despite this difference in mechanism, the combined failure rate approached 100% for observations where content was checked. This indicates that the problem is systemic rather than isolated to a single model architecture. Whether the model refuses to answer or guesses a wrong URL, the reliability of the citation remains compromised without a content-level review.

Why Smaller Entities Get More Ghost Citations

The frequency of ghost citations correlates strongly with the visibility of the entity being queried. Large multinational corporations have extensive digital footprints, making it easier for models to retrieve accurate citations. Smaller organizations, particularly those in Central and Eastern Europe, suffer from a lack of representation in training data. When the model encounters a query about a lesser-known entity, it cannot find a direct match in its internal weights. The system then defaults to the nearest available vector match.

This vector proximity leads to a substitution error. The model selects a well-known entity that shares linguistic features with the target entity. It then retrieves a real URL associated with that well-known entity. The link is valid, but the content describes the wrong organization. This mechanism explains why regional businesses face higher risks of misattribution. The training corpus contains fewer documents about these entities, forcing the model to bridge the gap with existing, but incorrect, data points.

The phenomenon is not random noise but a structural limitation of current language models. The model does not fabricate from malice; it fabricates from topology. It fills the void of missing information with the closest available pattern. For a user, this looks like a confident citation. For the entity, it looks like a loss of identity. The AI effectively erases the specific details of the small business and replaces them with the generic profile of a larger competitor.

This dynamic creates a specific vulnerability for due diligence and compliance workflows. Financial institutions and legal teams often rely on AI to summarize background information on niche suppliers or local partners. If the system substitutes a clean, large corporation for a risky, small one, the verification fails completely. The ghost citation provides a false sense of security, masking the true nature of the entity under review. The risk is highest where digital documentation is sparse and the model's confidence is artificially inflated by proximity.

Whoever Stands Closest in Vector Space

The mechanism behind ghost citations is rooted in how neural networks represent knowledge. Entities are mapped as points in a high-dimensional space. When a query is made, the model calculates the distance between the query vector and the stored vectors of known entities. If the target entity is far away or absent, the model selects the nearest neighbor. This neighbor is often a different entity that shares similar keywords or industry classifications. The model then generates text based on this neighbor's data.

This process results in a seamless transfer of attributes. The AI takes the URL of the neighbor and presents it as the source for the target. The user sees a link to a legitimate website. The text on that website matches the description of the neighbor, not the target. Because the neighbor is real, the link works. Because the text is coherent, the user assumes the connection is valid. The error lies in the attribution, not the existence of the resource.

The study confirmed that this behavior is consistent across different prompting strategies. Changing the tone of the request or asking for more details did not reduce the error rate. The substitution happens because the model lacks the specific token sequence for the target entity. It cannot hallucinate a new entity from scratch; it must rearrange existing tokens. The closest arrangement is the neighbor entity. This constraint forces the model to provide a real link to a wrong place.

Understanding this vector behavior helps explain why feedback loops fail. Telling the AI that the link is wrong does not force it to find the correct one. The model simply finds another neighbor or repeats the same substitution with a slightly different phrasing. The topology of the training data dictates the output, not the user's correction. The only way to avoid this is to prevent the model from attempting to cite entities it does not know, rather than trying to correct the citation after it is generated.

What a Link Check Cannot Tell You

A standard link check verifies connectivity and server status. It confirms that the URL is reachable and returns a valid HTTP code. It does not verify the semantic content of the page. It does not confirm that the text on the page matches the claim made by the AI. This limitation renders simple link checks insufficient for verifying AI outputs. The tool confirms the path but ignores the destination. Users who rely on this metric are left with a false sense of validation.

The study demonstrated that 100% of the observations with content checks contained at least one source failure. This included both dead links and ghost citations. The presence of a working link did not correlate with the accuracy of the information. In many cases, the working link led to a page that explicitly contradicted the AI's statement or described a completely different subject. The link check passed, but the verification failed. This gap represents the primary risk in automated trust systems.

To mitigate this risk, users must adopt content-aware verification habits. This involves reading the source page to confirm the entity name, the date, and the specific claim. Automated tools that scrape the page content and compare it to the query can also help. These tools look for keyword overlap and entity matching rather than just URL validity. Without this layer of inspection, the user remains vulnerable to the subtle deception of the ghost citation.

The implications extend to the design of AI systems themselves. Developers building retrieval-augmented generation pipelines must account for this failure mode. A system that retrieves documents and cites them must verify that the retrieved document actually supports the generated text. Relying on the assumption that a retrieved link is relevant is a critical architectural flaw. The system must include a content validation step before presenting the citation to the user. Otherwise, the pipeline propagates errors with high confidence.

AI-assisted: drafted with a language model from the author's research notes, then edited and checked before publication. Author: Zoltán Varga, AI visibility researcher and AI ecosystem architect, Budapest.