Getting a page into Google is a three-stage pipeline: crawl, index, serve. Crawling means Googlebot fetches the URL. Indexing means Google analyses the content and stores it in its database. Serving means Google picks that stored entry to show in results. Most people think crawling automatically leads to indexing. That's the first misconception. In reality, a page can be crawled and never indexed. Understanding the difference is the first step to fixing indexing problems.
This article corrects five common misconceptions about crawling and indexing. I'll explain what each stage actually does, show you how to check your own site, and give you a self-check list you can run today. The goal is to help you stop chasing the wrong problem and focus on what actually gets pages indexed.
How to tell crawling, indexing and serving apart
Crawling is the discovery and fetching of a URL by a bot. Googlebot follows links from known pages or reads your sitemap to find new URLs. When it fetches a page, it downloads the HTML, CSS, JavaScript and images. This is the crawl stage. It doesn't mean Google has stored anything yet. It just means the bot visited and retrieved the content.
Indexing is the process of analysing the fetched content and adding it to Google's index. The index is a massive database of web pages that Google can serve in search results. During indexing, Google parses the text, extracts links, processes structured data, and evaluates quality signals. If Google decides the page is not worth indexing, it will be marked as 'Crawled - currently not indexed' or 'Discovered - currently not indexed'. Both mean the page is not in the index.
Serving is when Google selects a page from the index to show for a user's query. Even if a page is indexed, it may never be served. Indexing is not ranking. Ranking is a separate process that happens at query time. A page can be indexed but rank on page 10 or not at all. The index is the library; serving is the librarian choosing which book to hand you.
Put simply: crawling is the visit, indexing is the storage, serving is the display. Each stage has its own rules and failure modes. Fixing a crawling problem won't help if the issue is indexing.
How to decode Google Search Console statuses (and what they really mean)
Google Search Console (GSC) shows you the indexing status of your pages in the 'Pages' report under 'Indexing'. The statuses are not all the same. Some mean the page is in the index, some mean it's not. Here's how to read them:
| GSC Status | What it means | In the index? |
|---|---|---|
| Discovered - currently not indexed | Google knows the URL but hasn't crawled it yet. | No |
| Crawled - currently not indexed | Google crawled the page but chose not to index it. | No |
| Indexed, not submitted in sitemap | The page is indexed, but you didn't list it in your sitemap. | Yes |
| Submitted and indexed | The page is in the index and was found via your sitemap. | Yes |
| Duplicate without user-selected canonical | Google chose a different page as the canonical version. | No (for this URL) |
| Excluded by 'noindex' tag | You told Google not to index the page. | No |
The most misunderstood status is 'Crawled - currently not indexed'. People see 'crawled' and assume the page will soon be indexed. That's not how it works. Google crawled the page and decided not to index it. The decision is final until you change something. The page might be thin, duplicate, low-quality, or simply not valuable enough compared to other pages on the web.
Another common mistake is confusing 'Discovered' with 'Crawled'. Discovered means Google found the URL but hasn't fetched it yet. This often happens on small sites with limited crawl budget. Google knows the page exists but hasn't spent resources on it. If you see many pages stuck in 'Discovered', your crawl budget is likely the bottleneck.
How to fix 'Crawled - currently not indexed' (step by step)
When a page is crawled but not indexed, Google is telling you it saw the content and passed. The fix depends on why it passed. Here are the steps I would take, in order:
- Check for thin content. Does the page have unique, substantial information? If it's mostly boilerplate or a few sentences, Google may see it as low value. Add depth: original data, examples, or a clear answer to a specific question.
- Look for duplication. Use a site: search or a tool like Siteliner to find pages with similar content. If multiple pages cover the same topic, consolidate them or use canonical tags to point to a strong version.
- Improve internal linking. Pages that are linked from many other pages on your site are more likely to be indexed. Add contextual links from relevant, already-indexed pages.
- Check your sitemap. Make sure the page is included and that the sitemap is submitted in GSC. A clean sitemap helps Google prioritise.
- Review technical issues. Is the page blocked by robots.txt? Does it have a noindex tag? Is it loading slowly? Fix these before requesting indexing.
- Request indexing. In GSC, use the URL Inspection tool and click 'Request Indexing'. This puts the page in a queue for crawling, but it doesn't ensure indexing.
After making changes, wait. Indexing can take days or weeks. There is no public figure for how long it takes because it depends on your site's authority, crawl budget, and the quality of the change. If the page remains 'Crawled - currently not indexed' after a month, the issue is likely content quality or duplication, not technical.
How to improve crawl budget for small sites (without wasting time)
Crawl budget is the number of URLs Googlebot will crawl on your site in a given period. For small sites, crawl budget is limited. Google allocates crawl resources based on site authority, update frequency, and server health. If you have thousands of low-value URLs, Google may waste its budget on them and miss your important pages.
To improve crawl budget, start by reducing the number of low-value URLs. Block faceted navigation, session IDs, and internal search results in robots.txt. Use noindex for pages you don't want in the index. Keep your sitemap clean and only include canonical, indexable URLs. Fix broken links and redirect chains, because each redirect costs crawl budget.
Server speed matters. If your server responds slowly, Googlebot will crawl fewer pages. Aim for a response time under 200 milliseconds. Use a CDN if your audience is global. Also, update your content regularly. Sites that change often get crawled more often. A blog that publishes weekly will be crawled more frequently than a static site that never changes.
One myth is that submitting a sitemap ensures indexing. It doesn't. A sitemap is a hint, not a command. Google may still choose not to crawl or index a URL. Another myth is that crawl budget is only for large sites. Small sites have crawl budgets too, just smaller. If you have 50 pages and Google only crawls 10 a day, your budget is 10 URLs per day. That's plenty for most small sites, but if you add 500 pages at once, it will take weeks to crawl them all.
What to ignore: when indexing issues don't matter
Not every indexing issue needs fixing. If a page is not indexed and it's not important for traffic, leave it. For example, thank-you pages, internal search results, and tag archives often don't need to be indexed. They can even hurt by diluting your crawl budget. Focus on pages that target keywords and bring traffic.
Also, don't obsess over the exact number of indexed pages. Google's index is not a complete list of your site. Some pages may be indexed but not reported in GSC, and some may be indexed under a different canonical URL. The 'Pages' report is a guide, not a scorecard. What matters is whether your important pages are indexed and ranking.
If you have a small site with 10-20 pages and all your important pages are indexed, you don't have an indexing problem. You have a ranking or content problem. Don't waste time on indexing when you should be improving quality and building links.
How to use crawl stats to diagnose indexing issues
Google Search Console has a 'Crawl Stats' report under 'Settings'. It shows how many requests Googlebot made to your site, the average response time, and the crawl purpose. Use it to spot patterns. If the crawl request count drops suddenly, your site may have a server issue. If response time is high, Googlebot may slow down. If most crawls are for 'Discovery' rather than 'Refresh', Google is still finding new URLs.
Compare crawl stats before and after you make changes. If you add internal links and request indexing, you should see more crawl requests for those URLs. If you block low-value URLs in robots.txt, you should see fewer requests for them. This is how you verify that your fixes are working.
One caveat: crawl stats are delayed by a few days. Don't expect instant feedback. Also, the data is sampled, so it's not exact. Use it as a trend, not a precise measurement.
Self-check list: is your page actually indexed?
Before you panic about a page not being indexed, run this check:
- Search for the exact URL in Google. If it appears, it's indexed.
- Use the URL Inspection tool in GSC. It shows the index status for that specific URL.
- Check if the page is blocked by robots.txt or has a noindex tag.
- Look for canonical tags pointing to a different URL.
- Check if the page is in your sitemap and if the sitemap is submitted.
- Review the page content: is it unique and substantial?
- Look at internal links: are there links from other indexed pages?
- Check server response time and errors in the Crawl Stats report.
If you've done all this and the page is still not indexed, the issue is likely quality or authority. Building links and improving content is the long-term fix. There's no shortcut.
Key Facts at a Glance
- Crawling is fetching; indexing is storing; serving is displaying. They are separate stages.
- 'Crawled - currently not indexed' means Google saw the page and chose not to index it.
- Indexing is not ranking. An indexed page can still rank poorly or not at all.
- Crawl budget is limited for all sites, including small ones. Reduce low-value URLs to free up budget.
- Submitting a sitemap does not ensure indexing. It's a hint, not a command.
- Not every page needs to be indexed. Focus on pages that drive traffic.
What does 'crawled - currently not indexed' mean?
It means Googlebot fetched the page but decided not to add it to the index. The page is not eligible to appear in search results. Common reasons include thin content, duplication, or low perceived value. You can request indexing again after improving the page.
How long does it take for Google to index a page?
There is no fixed time. It can take days to weeks, depending on your site's authority, crawl budget, and content quality. New sites often wait longer. If a page is crawled but not indexed, waiting won't help; you need to fix the underlying issue.
Why is my page crawled but not indexed?
Google crawled it and chose not to index it. This usually happens because the content is thin, duplicate, or not valuable enough compared to other pages. Technical issues like slow loading or noindex tags can also cause it. Check the page quality and your crawl stats.
How can I improve crawl budget for my small site?
Reduce the number of low-value URLs by blocking them in robots.txt or using noindex. Keep your sitemap clean and submit it in GSC. Improve server response time and fix broken links. Publish new content regularly to encourage more frequent crawling.
What's the difference between crawling and indexing?
Crawling is when Googlebot fetches a URL and downloads its content. Indexing is when Google analyses that content and stores it in its database. A page can be crawled without being indexed. Indexing is necessary for a page to appear in search results, but it doesn't ensure ranking.