1URL Parameter Pollution

The problem: Your ad campaigns and social media posts add tracking parameters to URLs. Every unique URL looks like a separate page to Google.

Why it happens: Facebook, Google Ads, and other platforms automatically append tracking parameters. If you're running ads, you might have the same page crawled 50+ times with different parameters.

2Thin Content Pages

The problem: Tag pages, category archives, and paginated results often have minimal unique content.

Why Google ignores them: These pages typically just list links to other content. They don't provide unique value that isn't available on the linked pages themselves.

3Feed & Archive URLs

The problem: WordPress generates RSS feeds for everything—categories, tags, authors, comments. These get crawled but shouldn't be indexed.

RSS feeds are meant for feed readers, not search results. Google correctly identifies these as not index-worthy, but they still waste crawl budget.

4Duplicate Content Issues

The problem: The same content is accessible at multiple URLs, or pagination creates near-duplicate pages.

When Google sees multiple URLs with similar content, it picks one (the "canonical") and ignores the rest.

5Technical Files Being Crawled

The problem: JavaScript files, CSS files, and WordPress system files are being discovered and crawled.

These files are necessary for your site to function, but they're not pages that should appear in search results.

6Low-Quality Service Area Pages

The problem: Auto-generated location pages with minimal unique content.

Many businesses create dozens of service area pages that are essentially the same content with the city name swapped out. Google recognizes this pattern.

Update Your robots.txt

Add these rules to block crawling of problematic URL patterns:

# Block tracking parameters
Disallow: /*?fbclid=*
Disallow: /*?utm_source=*
Disallow: /*?utm_medium=*
Disallow: /*?utm_campaign=*
Disallow: /*?gclid=*
Disallow: /*?gbraid=*
Disallow: /*?gad_source=*

# Block feed URLs
Disallow: /*/feed/
Disallow: /feed/
Disallow: /comments/feed/

# Block WordPress system files
Disallow: /wp-includes/
Disallow: /wp-json/
Disallow: /wp-admin/

# Block thin archive pages
Disallow: /tag/*/
Disallow: /author/*/

# Block preview URLs
Disallow: /*?p=*

Note: robots.txt blocks future crawling but doesn't remove already-indexed pages. For that, you need noindex tags.

Add Canonical Tags

Canonical tags tell Google which version of a URL is the "official" one. For WordPress, use Yoast SEO or RankMath to:

• Strip tracking parameters from canonical URLs automatically
• Set category/tag archives to point to the main archive page
• Ensure pagination pages canonicalize properly

In Yoast: Go to SEO → Search Appearance → Taxonomies and configure canonical settings for tags and categories.

Use Noindex Tags

For pages you want crawled but not indexed, add noindex meta tags. In WordPress with Yoast:

• Tag archives: SEO → Search Appearance → Taxonomies → Tags → Show in search results: No
• Author archives: SEO → Search Appearance → Archives → Author archives: Disabled
• Date archives: SEO → Search Appearance → Archives → Date archives: Disabled

Configure Google Search Console URL Parameters

Tell Google how to handle URL parameters:

1. Go to Google Search Console → Settings → URL Parameters
2. Add each tracking parameter (fbclid, utm_source, gclid, etc.)
3. Set each to "No URLs" - tells Google these parameters don't change page content

Note: Google has deprecated this feature in the new GSC but legacy settings may still apply.

Clean Up Your Sitemap

Your sitemap should only include URLs you want indexed. In Yoast:

• Exclude tag archives from sitemap
• Exclude author archives
• Exclude any pages you've set to noindex
• Ensure only canonical URLs appear

After cleaning your sitemap, resubmit it in Google Search Console.

How Long Will This Take to Fix?

After implementing these fixes:

• 1-2 weeks:Google will start respecting new robots.txt rules
• 2-4 weeks:Noindex pages will start dropping from the index
• 1-3 months:"Crawled not indexed" count should significantly decrease