The Number One Crawl Bloat Source
Faceted navigation (e.g., e-commerce product filters for size, color, brand, price range) generates a nearly infinite number of URL permutations. If left unchecked, search engines will crawl millions of useless URLs (e.g., `?color=red&size=medium&sort=price_asc`), devastating crawl budget and causing keyword cannibalization across duplicate pages.
Strategies: Controlling the Crawl
You must actively prevent Google from crawling unhelpful filter combinations:
- robots.txt Disallow: Block crawling of specific parameters (e.g., `Disallow: /*?*sort=`) or multiple facet combinations.
- Controlled Internal Linking: The most robust defense. Convert the `href` of non-indexable filter buttons into a `
- rel="nofollow": Adding nofollow to facet links can help, but it's a weak signal compared to altering the link architecture.
Strategies: Controlling the Index
If facet URLs do get crawled, you must control whether they enter the index:
- Canonicalization: Point the canonical tag of a filtered page (e.g., `?color=red`) back to the main category page. This is best for facets that don't have search demand.
- Noindex: Apply a `noindex` tag to facet URLs. This is often preferred over canonicals when the facet drastically changes the page content, causing Google to ignore the canonical.
- Indexable Facets: Only open a facet to crawling and indexing if it maps to verifiable search volume (e.g., allowing `?brand=nike` to index as a dedicated "Nike Shoes" page, but blocking `?size=11`).