Status Codes Taxonomy
Crawlers operate on HTTP status codes long before looking at page content.
- 200 OK: Success. The page is indexable.
- 304 Not Modified: Crucial for crawl budget. Tells the bot the page hasn't changed since its last crawl, saving server resources.
- 301 / 308: Permanent redirect. Passes equity and eventually removes the old URL from the index.
- 302 / 307: Temporary redirect. Passes equity, but the old URL remains in the index longer.
- 404 Not Found: The page is gone. Crawlers will revisit it periodically to confirm before dropping it.
- 410 Gone: Explicitly tells engines the page was intentionally removed permanently. Processed faster than a 404.
- 503 Service Unavailable: The correct code to use during planned server maintenance. It tells Googlebot to back off and try again later without dropping indexation.
Redirect Chains and Loops
A redirect chain occurs when A redirects to B, and B redirects to C. Googlebot follows up to 5 redirects in a chain before giving up and failing the crawl. Chains dilute link equity, increase latency, and harm crawl efficiency. All legacy redirects must be flattened so A points directly to C. A redirect loop (A to B to A) causes an immediate hard crawl failure.
Soft 404s
A "Soft 404" is an SEO disaster. It happens when a page displays "Not Found" or "Product Out of Stock" text to the user, but the server returns a 200 OK status code. Because of the 200 code, Googlebot tries to index the error page. Alternatively, it can occur when a URL returns a 200 but has very little content, causing Google's algorithm to classify it as a 404. Always return a hard 404 or 410 for missing content.