John Mueller recently sparked an insightful discussion about a long-standing SEO headache—unwanted indexed URLs, particularly those with query parameters. His advice is gaining attention in the SEO community, challenging traditional fixes and offering a more strategic approach.
Hidden URLs Sneaking into Google’s Index
A recent SEO audit related post revealed a significant indexing issue affecting an e-commerce website. More than half of the site’s 1.43k indexed pages were unnecessary “add to cart” URLs and paginated pages.
These unwanted URLs were appearing in Google’s index despite the site using rel=canonical attributes to direct Google to the preferred URLs.
The audit demonstrated a common SEO challenge: rel=canonical is merely a suggestion, not a strict directive.
The URLs in question looked something like this:
example.com/product/page-5/?add-to-cart=example
Recognizing the issue, the SEO expert conducting the audit proposed using a meta noindex directive to de-index these pages and then blocking them in the robots.txt file.
Google Strikes Back: A Smarter, More Targeted Fix
John Mueller weighed in on the debate, offering an alternative approach that many experts found intriguing. Instead of treating the issue as a random list of URLs to be canonicalized, Mueller recommended analyzing URL patterns and applying tailored solutions.
He suggested:5
- Identifying URL patterns to understand why these pages were getting indexed in the first place.
- Blocking “add to cart” URLs directly in robots.txt, preventing Google from crawling them altogether.
- Exploring Google’s documentation on handling pagination and filtering in URL parameters for a more structured solution.
This approach reinforces Google’s philosophy that SEO strategies should be site-specific rather than relying on one-size-fits-all fixes.
Why Some URLs Refuse to Disappear
The problem of unwanted indexed URLs is not new. Many SEOs struggle with dynamically generated pages—such as faceted navigation pages and shopping cart URLs—being indexed against their wishes.
The issue arises because search engines don’t always interpret canonical tags as strict instructions.
Several LinkedIn commenters raised the question: Why is Google even indexing shopping cart URLs in the first place?
While there was no definitive answer, one theory is that certain shopping cart platforms might be inadvertently exposing these URLs to search engines. This suggests that businesses using e-commerce platforms should scrutinize how their systems handle query parameters.
How to Take Control of Your Indexed URLs
According to Mueller’s comment, SEO isn’t just about applying generic fixes—it’s about understanding how Google interacts with a site’s structure.
Some key takeaways include:
- Rel=canonical isn’t a guaranteed fix. Google treats it as a hint, meaning alternative indexing controls are sometimes necessary.
- Robots.txt is an effective tool. Preventing Google from crawling problematic URLs in the first place is often better than trying to fix the issue after the fact.
- SEO decisions should be data-driven. Patterns in indexing behavior can reveal underlying issues that require more tailored solutions.
E-Commerce Rescue Plan: What You Need to Do Now
For businesses struggling with similar issues, here’s a practical game plan:
Audit your indexed pages – Identify unwanted URLs and analyze indexing patterns.
Check canonical tags – Ensure they are correctly set up, but understand that they might not always work as expected.
Use robots.txt wisely – Block unnecessary URLs to prevent search engines from crawling them.
Review e-commerce platform settings – Ensure your system isn’t inadvertently exposing query parameters.
Follow Google’s documentation – Keep up with best practices to ensure a well-structured website.
Key Takeaways
- Google’s rel=canonical tag is a suggestion, not a strict rule.
- “Add to cart” URLs and faceted navigation pages should often be blocked via robots.txt.
- URL patterns must be analyzed to determine the root cause of indexing issues.
- E-commerce platforms may unintentionally expose unwanted URLs to search engines.
- SEO fixes should be tailored to each site’s structure rather than relying on generic solutions.
Dileep Thekkethil
AuthorDileep Thekkethil is the Director of Marketing at Stan Ventures, where he applies over 15 years of SEO and digital marketing expertise to drive growth and authority. A former journalist with six years of experience, he combines strategic storytelling with technical know-how to help brands navigate the shift toward AI-driven search and generative engines. Dileep is a strong advocate for Google’s EEAT standards, regularly sharing real-world use cases and scenarios to demystify complex marketing trends. He is an avid gardener of tropical fruits, a motor enthusiast, and a dedicated caretaker of his pair of cockatiels.