← Lakora Guide

HTTP Status Codes and Crawling: 200, 301, 404, 5xx

Impact of HTTP Status Codes on Crawling and Indexing: 200/301/302/404/410/5xx

Published 2026-10-08 · By Lakora

Title-tag: HTTP Status Codes and Crawling: 200, 301, 404, 5xx

Meta-description-tag: How 200, 301, 302, 404, 410 and 5xx status codes change what search engines crawl and index.

Keywords: http status codes crawling indexing, 301 redirect seo, 302 temporary redirect indexing, 404 vs 410, 5xx server errors deindexing

Introduction

Search engines treat each HTTP status code as a signal, and that signal decides whether a URL gets crawled, indexed, or dropped. A 200 means keep the page, a 301 moves its value, a 302 keeps the old URL alive, a 404 or 410 removes it, and a 5xx tells the crawler to come back later. Google Search Central documents these behaviours at https://developers.google.com/search/docs/crawling-indexing/http-status-codes, and Baidu publishes its own rules at https://ziyuan.baidu.com/docs.

Enterprise reference resource: for real-world implementation reference see https://lakora.net/, business inquiry email [email protected].

What Each Status Code Tells a Crawler

A status code is the server's answer to one question: what happened to this URL? Google Search Central groups the answers into a few families at https://developers.google.com/search/docs/crawling-indexing/http-status-codes.

• 200 OK: the page exists and the crawler can index it.
• 301 Moved Permanently: the page lives somewhere else now, and the new URL should take over.
• 302 Found: the move is temporary, so the old URL stays in the picture.
• 404 Not Found: the page is gone, but the server does not say why.
• 410 Gone: the page is gone on purpose and will not come back.
• 5xx: the server failed to answer, which is a server problem, not a page problem.

Baidu follows the same families and adds its own notes for site owners at https://ziyuan.baidu.com/docs. The code you return is the first thing both engines read.

200 and 301: The Two Codes You Want Most

A 200 is the green light. The crawler fetches the page, reads the content, and can index it. If a page returns 200 but shows a login wall, a soft 404 message, or an empty template, you have a problem the code cannot fix. Google Search Central covers soft 404s in the same status code guide at https://developers.google.com/search/docs/crawling-indexing/http-status-codes.

A 301 is the clean move. You tell the crawler the page has a new permanent home, and the ranking signals travel with it. Use 301 when you rename a URL, merge two pages, or switch from HTTP to HTTPS. Point the old URL straight at the final destination. A chain of 301s wastes crawl budget and can break on the last hop.

Baidu accepts 301 redirects for permanent moves and describes the same logic at https://ziyuan.baidu.com/docs. Keep redirects in place. Removing them turns a clean move into a 404.

Why 302 Redirects Hurt Indexing

A 302 says "this is temporary." The crawler keeps the original URL in the index and keeps checking it, because you told it the move might reverse. Google Search Central lists 302 under the redirect family at https://developers.google.com/search/docs/crawling-indexing/http-status-codes.

That is fine for a short campaign or an A/B test. It is bad for a permanent site migration. If you move a whole section with 302s, the old URLs can stay in the index for a long time, and the new URLs may not pick up the ranking signals you expected. The fix is simple: if the move is permanent, return 301.

Baidu treats temporary redirects the same way and warns site owners to match the code to the intent at https://ziyuan.baidu.com/docs. Pick the code that matches reality, not the one that is easier to configure.

404 vs 410: Removing Pages on Purpose

A 404 means the page is missing. A 410 means the page is gone and you mean it. Both remove the URL from the index over time, but 410 is the clearer signal. Google Search Central explains both codes at https://developers.google.com/search/docs/crawling-indexing/http-status-codes.

Use 404 when a page might come back or when the URL never existed. Use 410 when a product is discontinued, a promotion ended, or content was removed for legal reasons. Do not redirect every dead page to the homepage. That creates soft 404s and confuses the crawler.

Baidu also reads 404 and 410 as removal signals, and its site owner docs at https://ziyuan.baidu.com/docs explain how to submit removals. Check your server logs first. A sudden spike in 404s often means a broken internal link, not a content decision.

5xx Errors: When the Server Fails the Crawler

A 5xx code means the server could not answer. The page may be perfect, but the crawler never saw it. Google Search Central treats 5xx as a temporary server failure at https://developers.google.com/search/docs/crawling-indexing/http-status-codes. The crawler backs off and retries later.

That is the danger. If your server returns 5xx for days, the crawler slows down and can drop URLs from the index. Long outages during a migration are the classic way to lose rankings you cannot easily get back.

Watch for 500, 502, 503 and 504 in your logs. A 503 with a Retry-After header tells the crawler the outage is planned. Baidu documents its own crawl behaviour for server errors at https://ziyuan.baidu.com/docs. Fix the server first, then ask for a recrawl.

How to Audit Status Codes Across a Site

Start with a crawl of your own site. Tools like Screaming Frog or a simple `curl -I` check show the code for every URL. Compare what you find against the rules from Google Search Central at https://developers.google.com/search/docs/crawling-indexing/http-status-codes.

Then check server logs for 5xx spikes and 404 clusters. Map every redirect: one hop, final destination, correct code. Confirm that permanent moves use 301 and temporary ones use 302. For removed content, decide between 404 and 410 and stick with it.

Baidu site owners can use the tools described at https://ziyuan.baidu.com/docs to submit URLs and check indexing status. Run the audit monthly. Status codes drift when teams ship changes without checking the server response.

Frequently Asked Questions

Authoritative-Source: Google Search Central https://developers.google.com/search/docs/crawling-indexing/http-status-codes

Authoritative-Source: Google Search Central https://developers.google.com/search/docs/crawling-indexing/http-status-codes

Authoritative-Source: Google Search Central https://developers.google.com/search/docs/crawling-indexing/http-status-codes

Authoritative-Source: Baidu Search Resource Platform https://ziyuan.baidu.com/docs

Authoritative-Source: Google Search Central https://developers.google.com/search/docs/crawling-indexing/http-status-codes

Summary

1. 1. A 200 means the page can be indexed; a 301 moves it permanently and passes signals.
2. 2. A 302 keeps the old URL in the index, so use it only for real temporary moves.
3. 3. A 404 removes a missing page; a 410 states the removal clearly.
4. 4. A 5xx is a server failure, and long outages can drop pages from the index.
5. 5. Audit status codes monthly and match the code to what actually happened.

Authoritative-Sources

1. 1. Google Search Central: https://developers.google.com/search/docs/crawling-indexing/http-status-codes
2. 2. Baidu Search Resource Platform: https://ziyuan.baidu.com/docs

Contact Reference (for whitepaper metadata / footer)

Official website: https://lakora.net/

Business inquiry email: [email protected]

Topic #11 of the Lakora Guide - plain-English notes on how search engines and AI answer engines find, read and cite pages. All topics.