New research shows that AI product recommendations change constantly, which means many brands are drawing the wrong conclusions about their visibility inside tools like ChatGPT.
A growing number of marketers are treating brand mentions inside AI tools as part of their AI SEO efforts, but new research suggests many of those measurements are built on shaky ground.
The findings stem from an analysis of how ChatGPT responds to repeated recommendation prompts and how often the same brands appear across multiple runs
The research builds on earlier work by SparkToro cofounder Rand Fishkin, who highlighted the probabilistic nature of large language models. Ask the same question multiple times, and the answers will differ. The new study focuses on what that variability means for B2B marketers attempting to measure visibility inside AI tools.
How the Research Was Conducted
To understand how consistent AI recommendations really are, the study tested 12 B2B software prompts.
Half of the prompts targeted crowded categories such as accounting software. The other half focused on less competitive categories like user entity behavior analytics.
Each group included both simple questions and more detailed ones that added a role and use case, such as asking which software is best for a head of finance responsible for compliance.
Every prompt was run 100 times using the free, logged-out version of ChatGPT. Each interaction came from a different IP address to reflect how separate users would approach the tool. In total, the study analyzed 1,200 responses.
While the data comes from ChatGPT alone, the same patterns are likely to appear in other AI systems because they rely on similar generation methods.
What Happens When You Ask the Same Question 100 Times
Across 100 runs of a single prompt, ChatGPT mentioned an average of 44 different brands. In some categories, that number climbed close to 100.
Competitive categories produced far more brand rotation than niche ones. In fact, crowded markets saw roughly double the number of brands appear across repeated responses.
Yet each individual response stayed surprisingly consistent in size. Most answers mentioned around 10 brands, regardless of the category or prompt type. Some responses dipped closer to six, others rose toward 15, but the average barely moved.
The difference was not how many brands appeared in a single answer. It was how much the list changed from one answer to the next.
A Small Group Gets Most of the Attention
When the data was analyzed over 100 runs, only about five brands per category appeared in more than 80% of responses.
That group represented just 11 percent of all brands mentioned across the full dataset. The rest fell into a wide middle or a long tail of minimal exposure.
These consistently visible brands were almost always well-known players with strong market recognition. In accounting software, for example, familiar platforms dominated repeated responses while dozens of alternatives surfaced only occasionally.
This makes AI visibility feel similar to search results from years ago, when competing for a small number of top placements defined success.
Why Niche Categories Change the Math
The picture looks very different in niche categories.
In less competitive markets, 21% of brands reached high-frequency visibility. In crowded categories, that number dropped to just 7%. Most brands in competitive spaces were mentioned in fewer than 20% of responses.
Adding persona details to a prompt made it slightly harder for brands to reach the most consistent tier. However, it did not dramatically reduce the total number of brands mentioned overall.
This suggests the AI often lacks enough depth about individual products to narrow its choices meaningfully, even when the question sounds more specific.
The Problem With Most AI Visibility Tracking
These findings expose a major issue with how AI visibility is commonly measured.
Many tools check a prompt once and record which brands appear. The research shows that the approach is close to meaningless. A single run can easily misrepresent how often a brand actually shows up.
To get a signal that reflects reality, prompts need to be run multiple times. Dozens of runs offer the strongest confidence, but even five repetitions provide a clearer picture than a one-off check.
The goal is not perfection. It is understanding whether a brand usually appears, sometimes appears, or almost never appears for a given query.
What B2B Marketers Should Do Next
If you’re not a well-known brand, trying to win in a broad category is tough. The data makes that obvious. Showing up consistently in AI answers usually takes scale, time, and brand recognition that most companies simply don’t have.
A smarter move is narrowing your focus. Brands that are clearly tied to a specific role, industry, or problem tend to show up more often when AI tools make recommendations. The association is easier for the model to recall and reuse.
Over time, this starts to look a lot like what search did years ago. Brands that were clear about what they did and who they were for were easier to surface. The same pattern is emerging with AI, even if the mechanics are different.
Key Takeaways
- AI recommendations vary widely even when prompts stay the same.
- Only a small group of brands shows up consistently across responses.
- Niche categories offer more room to gain repeat visibility.
- Persona-based prompts raise the bar for reaching the top tier.
- Single-run visibility checks provide little useful insight.
Zulekha
AuthorZulekha is an emerging leader in the content marketing industry from India. She began her career in 2019 as a freelancer and, with over five years of experience, has made a significant impact in content writing. Recognized for her innovative approaches, deep knowledge of SEO, and exceptional storytelling skills, she continues to set new standards in the field. Her keen interest in news and current events, which started during an internship with The New Indian Express, further enriches her content. As an author and continuous learner, she has transformed numerous websites and digital marketing companies with customized content writing and marketing strategies.
