New Study Reveals AI Assistants Hallucinate Links Nearly 3X More Than Google Search

AI hallucinations are no longer just about facts or made-up citations. A new study from Ahrefs explored a different and equally important question: how often do AI assistants hallucinate links and send users to non-existent pages?

The results are striking. After analyzing 16 million unique URLs cited by ChatGPT, Claude, Perplexity, Copilot, Gemini and Mistral, researchers found that AI assistants send users to 404 pages almost three times more often than Google Search does.

That’s a big deal for businesses and SEOs. If AI is changing how people discover content, what does it mean when the very links driving traffic sometimes lead to dead ends?

What Did the Study Examine?

The research, led by Ryan Law and Xibeijia Guan at Ahrefs, used two datasets to measure hallucination rates:

1. Web Analytics test (clicked URLs): Examined anonymized data from real websites to track visits referred by AI assistants.
2. Brand Radar test (cited URLs): Analyzed millions of URLs AI assistants cited in outputs, whether or not they were clicked.

Together, these tests created one of the most comprehensive snapshots to date of how AI-generated referrals actually behave online.

Test 1: Clicked URLs in Web Analytics

For the first test, Ahrefs used anonymized data from its Web Analytics tool, focusing on URLs referred by AI assistants. Pages were flagged as 404s if their titles contained “404” or “not found.”

The results?

Now compare that to Google, which had a baseline referral 404 rate of just 0.15% across 629 million URLs.

In other words, ChatGPT sends users to broken links 7X more often than Google. Even the better-performing AI tools (like Gemini and Perplexity) still exceed Google’s 404 rate.

Test 2: Cited URLs in Brand Radar

The second test used Brand Radar, Ahrefs’ searchable database of millions of AI prompts and outputs, to examine cited URLs. This went deeper than clicks by looking at everything the assistants recommended.

Here’s what they found:

Key insight:

• ChatGPT again tops the list, with a hallucination rate nearly 3X higher than its competitors.
• Perplexity and Gemini hover around 0.86–0.87%, almost identical to the 0.84% baseline 404 rate in Google’s SERPs. That makes sense, since both tools lean heavily on Google’s index.
• Copilot, which draws from Bing, shows a lower rate (0.54%).

Why Do AI Assistants Hallucinate Links?

Ahrefs identified two primary reasons:

1. Expired but once-valid URLs: Some links cited by AI were once real pages but have since been deleted or moved without redirects. Since AI blends internal knowledge with live search, it may recall URLs that no longer exist.
2. True “pattern-based” hallucinations: AI often generates URLs that look right but were never real. For example, Ahrefs noticed hallucinations like /blog/internal-links/ or /blog/newsletter/ on its own site. Both match plausible blog structures but don’t exist.

This is why hallucinations are not random. They are pattern-driven guesses based on how URLs usually look. For sites with structured naming conventions, this is both a curse (more hallucinations) and an opportunity (predictable traffic to fix or redirect).

How Big of a Problem Is This?

Here is where things get nuanced.

• On average, AI referrals make up just 0.25% of a website’s traffic.
• By comparison, Google drives nearly 40% of traffic.
• Even if 1% of AI referrals hit 404s, that’s 1% of 0.25% — a drop in the bucket compared to overall traffic.

So, no, hallucinated URLs aren’t going to sink your site tomorrow. But they represent:

• Lost opportunities: Even small volumes of misdirected traffic could be recaptured.
• Signals for the future: As AI adoption grows, today’s small problem could become tomorrow’s bigger one.

Remember: in 2023, almost no one thought AI would account for a measurable share of search traffic. By mid-2025, AI referrals already exist, and platforms like ChatGPT are experimenting with default web-browsing modes.

How Can Sites Detect Their Hallucinated URLs?

Ahrefs suggests a clear process for identifying hallucinated links:

• Filter analytics for AI traffic

Start by isolating traffic coming from AI assistants. In GA4, this can be done using a regular expression filter in the Session Source dimension, targeting patterns like .*gpt.*|.*chatgpt.*|.*perplexity.*|.*claude.*|.*gemini.*|.*copilot.*. If you are using Ahrefs’ Web Analytics, the process is even simpler; just select the built-in “AI search” channel filter to instantly see AI referrals.

• Export referral URLs and check their status codes

Once you have pulled your AI traffic data, the next step is to verify whether those URLs actually exist. You can use an Apps Script in Google Sheets to fetch HTTP status codes for each URL, or run a site crawl with your preferred SEO tool. This will reveal which pages return a 200 (valid), 301 (redirected), or 404 (dead) status.

• Prioritize URLs with meaningful traffic

Not every hallucinated link needs urgent attention. Focus on URLs that both return a 404 error and receive a noticeable volume of visitors — for example, anything with more than 10 visits per month. This helps you concentrate on the broken links that actually impact user experience and site metrics.

• Redirect hallucinated pages where possible

If the hallucinated URL resembles a real topic (say, /blog/keywords/), you can redirect it with a 301 to the closest relevant resource on your site. This ensures that even mistaken referrals send users to helpful content, recovering some of the lost value.

• Optimize your 404 page for stranded visitors

In cases where a redirect is not possible, don’t waste the traffic. Update your 404 page to include links to popular guides, categories, or newsletter sign-ups. This transforms a dead end into a discovery point, allowing you to re-engage visitors who landed there through AI-generated errors.

Should SEOs and Marketers Worry?

Right now, the honest answer is: not much, but don’t ignore it.

• Short-term: The impact is minimal. AI referrals are still niche compared to Google.
• Medium-term: Worth monitoring. With AI assistants already integrated into Bing, Google and upcoming Apple search products, traffic could scale quickly.
• Long-term: Potentially critical. If AI search becomes mainstream, link hallucinations could damage brand trust and visibility if left unaddressed.

Ahrefs itself is already adding a “hallucinated URL filter” to its analytics, signaling growing demand from webmasters to track this.

What Does This Mean for SEO?

Here’s the real takeaway: AI search does not behave like Google.

• Google: Shows ranked, verified URLs pulled from its index.
• AI: Generates conversational answers and may fabricate URLs to appear complete.
• Visibility ≠ rankings anymore. Being cited in AI results requires both brand authority and predictable URL structures.
• Site hygiene matters. Dead pages, poor redirects, or weak brand presence can amplify hallucinations.
• Prepare for audience filtering. As AI integrates guardrails (such as parental controls), what users see may differ based on age or context, shaping when and how your content is surfaced.

Dileep Thekkethil

Author

Dileep Thekkethil is the Director of Marketing at Stan Ventures, where he applies over 15 years of SEO and digital marketing expertise to drive growth and authority. A former journalist with six years of experience, he combines strategic storytelling with technical know-how to help brands navigate the shift toward AI-driven search and generative engines. Dileep is a strong advocate for Google’s EEAT standards, regularly sharing real-world use cases and scenarios to demystify complex marketing trends. He is an avid gardener of tropical fruits, a motor enthusiast, and a dedicated caretaker of his pair of cockatiels.