How to Fix Crawl Errors Reported in Google Search Console

To fix crawl errors in Google Search Console, open the Pages report under Indexing, click into each error type (like "Not found (404)", "Server error (5xx)", or "Blocked by robots.txt"), export the affected URLs, and fix the root cause: repair or redirect broken links, remove noindex or robots blocks from pages you want indexed, and clear server timeouts. Then click Validate Fix so Google recrawls and confirms. Most crawl errors come down to a handful of predictable causes, and once you know the pattern, each one takes minutes to resolve.

Where crawl errors actually live in Google Search Console

Google retired the old "Crawl Errors" report years ago. Today the information is split across two places, and knowing which report to check saves you a lot of confusion.

• Pages report (Indexing > Pages): This is your main dashboard. It groups every URL Google tried to crawl into "Indexed" and "Not indexed" buckets, with a reason for each non-indexed URL. This is where you spend 90% of your time.
• URL Inspection tool: Paste any single URL to see exactly how Googlebot sees it, when it was last crawled, and why it is or isn't indexed. Use this to debug one page at a time.
• Crawl Stats report (Settings > Crawl stats): Shows server response codes, host status, and crawl volume over time. Use this when you suspect a hosting or server-wide problem.

If you haven't connected your site yet, start with our guide on how to set up Google Search Console for a new website. You can't fix what you can't see.

The crawl and indexing errors you will actually see (and what each means)

Google uses specific status labels. Here is what they mean in plain English and how urgent each one is.

How to fix server errors (5xx) reported in Search Console

A 5xx means Googlebot asked for a page and your server choked. If this spikes, treat it as urgent because Google will slow down crawling of your whole site to avoid making things worse.

1. Reproduce it. Open the URL in the URL Inspection tool and click "Test Live URL." If it fails there too, the problem is live right now.
2. Check Crawl Stats. Go to Settings > Crawl stats and look at "Host status." A red mark points to a server-wide issue, not a single page.
3. Look for timeouts. Slow database queries, an overloaded shared host, or a plugin gone rogue are the usual culprits. Ask your host to check server logs for the timestamps Google reported.
4. Check for rate limiting. Some security plugins and CDNs (Cloudflare, Sucuri) throttle or block Googlebot by mistake. Whitelist Google's crawler.
5. Fix and validate. Once resolved, hit "Validate Fix" in the report. Google recrawls a sample and clears the error if the fix holds.

Intermittent 5xx errors that clear on their own are usually a capacity problem. If your host can't handle Googlebot's crawl rate, upgrade the plan or add caching.

How to fix "Not found (404)" crawl errors

Not every 404 is a problem. A 404 for a page you deliberately deleted and that nobody links to is fine. Google will eventually drop it. You only need to act when the 404 is on a URL that should exist or that other pages still link to.

Step 1: Export the URLs and sort by cause

Click into the "Not found (404)" row, export the list, and group the URLs into three buckets:

• Pages you moved or renamed: These need a 301 redirect to the new URL.
• Pages you deleted on purpose: Leave them as 404 (or 410 to signal permanent removal). Don't redirect them to the homepage in bulk, Google treats mass homepage redirects as soft 404s.
• Typos and broken internal links: Fix the source link so Google (and users) stop hitting the dead URL.

Step 2: Find where the broken links come from

In the URL Inspection tool, the "Referring page" data (when available) shows where Google found the link. For internal links, a full crawl is faster. Running a proper technical SEO audit for a small business website will surface every internal link pointing to a dead URL so you can fix them at the source.

Step 3: Redirect the ones worth saving

Set up 301 redirects from the old URL to the most relevant live page. Redirect to a genuinely equivalent page, not a generic category, or Google may flag it as a soft 404. This also matters for link equity: broken URLs with backlinks pointing at them are leaking authority, which ties directly into how you reclaim value through broken link fixes.

How to fix soft 404 errors

A soft 404 is sneaky: the page returns a 200 "OK" status, but Google decides the content is thin, empty, or says "nothing found." Common causes and fixes:

• Empty category or search results pages: Add content, or return a real 404 when there are no results.
• Thin pages with almost no text: Flesh them out or noindex them if they aren't meant to rank.
• "Product unavailable" pages that stay live: Either restore useful content or return a proper 404/410.
• Redirects to irrelevant pages: Point the redirect somewhere relevant instead.

The fix is almost always "make the page substantial" or "return the correct status code." After fixing, validate.

How to fix redirect errors and redirect chains

Redirect errors mean Googlebot got stuck: a loop (A redirects to B redirects back to A), a chain that's too long, or a redirect pointing to a URL that itself errors out.

1. Trace the chain. Use the URL Inspection tool or a redirect checker to map every hop.
2. Collapse chains to a single hop. If A goes to B goes to C, change A so it points directly to C.
3. Kill loops. Loops usually come from conflicting rules (for example, an HTTP-to-HTTPS rule fighting a www rule). Pick one canonical version of your domain and enforce it once.
4. Update internal links. Link directly to the final URL so you're not relying on redirects at all.

How to fix "Blocked by robots.txt" and 403 errors

These are access problems. Google wanted in and couldn't get in.

Blocked by robots.txt: Open your robots.txt file (yoursite.com/robots.txt) and look for a Disallow rule matching the blocked URL. If the page should be indexed, remove or narrow the rule. Test changes in the robots.txt Tester before pushing live. A single overly broad line like Disallow: / can wipe your whole site out of the crawl, so double check it.

Blocked due to access forbidden (403): Your server is refusing Googlebot. Causes include IP or user-agent blocking in a firewall or security plugin, hotlink protection, or a login wall on a page that should be public. Whitelist Googlebot and make sure public pages don't require authentication.

One important distinction: robots.txt blocks crawling, while a noindex tag blocks indexing. If you want a page gone from Google, use noindex and let Google crawl it. If you block it in robots.txt, Google can't see the noindex and may keep a bare listing.

How to handle "Crawled - currently not indexed" and "Discovered - not indexed"

These aren't crawl errors in the traditional sense, but they show up in the same report and confuse people, so let's clear them up.

Crawled - currently not indexed: Google saw the page and decided not to index it. This is almost always a quality or duplication signal. Improve the content depth, make sure it isn't a near-duplicate of another page, and strengthen internal links to it. If several pages compete for the same term, you may have a case of keyword cannibalization to resolve first. For pages stuck here, our guide on fixing pages that are indexed but not ranking covers the content side in depth.

Discovered - currently not indexed: Google knows the URL exists but hasn't crawled it yet, often a crawl-budget or priority issue on larger sites. Fix it by adding the page to your XML sitemap, linking to it internally from strong pages, and requesting indexing. Speed things up with our steps to get a new post indexed faster.

Your crawl error fix checklist

• Open Indexing > Pages and note every "Not indexed" reason with a count above zero.
• Prioritize by impact: 5xx and unintended robots/403 blocks first, then 404s and redirects, then soft 404s and content issues.
• Export affected URLs for each error type before you touch anything.
• Confirm the status code with the URL Inspection "Test Live URL" button.
• Apply the right fix: redirect, repair the link, remove the block, add content, or fix the server.
• Update internal links pointing to broken or redirected URLs.
• Resubmit your sitemap if you changed a lot of URLs.
• Click Validate Fix and check back in a few days.

Crawl errors aren't a one-time cleanup. New broken links, plugin updates, and site migrations create fresh ones constantly. Check the Pages report monthly, and pair it with ongoing rank tracking and site auditing in RankLoco so you catch a crawl problem before it drags your rankings down. If you want a broader tune-up, browse more SEO guides to keep the whole site healthy.

Frequently Asked Questions

How long does it take Google to clear crawl errors after I fix them?

After you click Validate Fix, Google usually starts recrawling a sample within a few days and updates the report over the following one to two weeks. Large sites take longer. The status shows "Validation started," then "Passed" once Google confirms the fix on enough URLs.

Do 404 errors hurt my Google rankings?

A normal 404 for a page you deleted does not hurt rankings, Google expects some 404s on every site. Problems arise when important pages return 404s, when internal links point to dead URLs, or when a mass of 404s signals a broken site. Fix the ones on URLs that should exist and ignore the rest.

What is the difference between a crawl error and an indexing error?

A crawl error means Googlebot couldn't successfully fetch the URL (server error, 403, robots block, timeout). An indexing issue means Google fetched the page fine but chose not to index it (thin content, duplication, noindex tag). Both appear in the Pages report but need different fixes.

Should I redirect all my 404 pages to the homepage?

No. Google treats bulk redirects to the homepage as soft 404s and may not pass any value. Redirect a deleted page only to a genuinely relevant replacement. If there is no good match, let the URL return a clean 404 or 410 and remove internal links to it.

Why does Search Console show pages blocked by robots.txt that I never blocked?

Check whether a plugin, CDN, or staging configuration added rules to your robots.txt, or whether a wildcard pattern is catching URLs unintentionally. A rule like Disallow: /*? can block every parameter URL. Test your robots.txt in the tester and narrow overly broad Disallow lines.

How often should I check for crawl errors in Search Console?

Check the Pages report at least monthly for a small site, and weekly during or right after a migration, redesign, or major content update. Set aside time whenever organic traffic dips, since crawl and indexing problems are a common cause worth ruling out early.