On this page · 8 sections
- 01What 'Discovered - currently not indexed' means
- 02Crawled vs discovered: the key difference
- 03Why Google postpones the crawl
- 04Crawl budget on large and programmatic sites
- 05Server speed and crawl capacity
- 06Internal links and sitemaps that signal priority
- 07How to get the pages crawled
- 08Tracking progress until they're indexed

Until Google crawls a page, it can't index it, rank it or use it anywhere. AI answers cite what's been crawled, and they lean toward pages that are kept current. FreshlyCited helps Google discover and index the pages, posts and backlinks you publish or update sooner, so they become eligible for Google Search, AI Overviews and AI Mode, then re-checks and shows you what Google has indexed. This guide covers what the status means, how it differs from "crawled - currently not indexed", why Google postpones crawls, and how to get the pages crawled and track them until they're indexed.
What 'Discovered - currently not indexed' means
"Discovered - currently not indexed" is a reason in the Search Console Page indexing report for URLs Google has found but not yet fetched. According to Google's Page indexing report documentation, "Typically, Google wanted to crawl the URL but this was expected to overload the site; therefore Google rescheduled the crawl."
Google usually finds these URLs through your sitemap, internal links or links from other sites. Finding a URL puts it in the queue. Crawling it is a separate step, and Google schedules it based on how much it wants the page and how much load your server can take.
Practically, this means:
- The last crawl field in URL Inspection will be empty or say the URL hasn't been crawled.
- Nothing about the page's content has been evaluated yet.
- The status often clears on its own for a small number of new URLs.
Crawled vs discovered: the key difference
The key difference is where the page stalled. "Discovered" pages are waiting to be crawled; "crawled" pages were fetched and then not indexed.
| Discovered - currently not indexed | Crawled - currently not indexed |
|---|---|---|
Has Googlebot fetched the page? | No | Yes |
Has Google judged the content? | No | Yes |
Typical cause | Crawl scheduling: priority, site capacity, crawl budget | Content value: thin, duplicate, low-value, weak signals |
Main fix | Make the URL easier and more important to crawl | Make the page more worth indexing |
Does a recrawl request help? | Often, because the page hasn't been seen | Only after the page has changed |
If your pages have already been fetched and rejected, the fixes are different. See crawled - currently not indexed for that case.
Why Google postpones the crawl
Google postpones a crawl when it decides the URL isn't worth crawling right now or that crawling more of your site would strain your server. Both sides of that equation are under your influence.
Common reasons:
- Server strain. Slow responses, timeouts or 5xx errors make Google crawl less to avoid overloading you.
- Too many low-value URLs. If Google keeps finding parameter variants, faceted pages or duplicates, it spends its crawl on those and deprioritizes the rest. Duplicates that Google has already processed show up under duplicate without user-selected canonical or alternate page with proper canonical tag.
- Weak priority signals. URLs that appear only in a sitemap, with no internal links, look less important than pages linked from your homepage or hubs.
- A sudden spike in new URLs. Publishing thousands of pages at once creates a queue Google works through at its own pace.
- A new or little-known site. Google's crawl budget guide notes that "URLs that are more popular on the Internet tend to be crawled more often." A site with few links has less crawl demand.
On a small site with a handful of new posts, reason 3 or 5 is far more likely than a true crawl budget limit.
Crawl budget on large and programmatic sites
Crawl budget is the number of URLs Google can and wants to crawl on your site. Google says its crawl budget guide is mainly for:

- "Large sites (1 million+ unique pages) with content that changes moderately often (once a week)"
- "Medium or larger sites (10,000+ unique pages) with very rapidly changing content (daily)"
- "Sites with a large portion of their total URLs classified by Search Console as Discovered - currently not indexed"
Google calls those numbers "a rough estimate," not exact thresholds. The last bullet matters: a high share of discovered-but-not-crawled URLs is itself Google's sign that crawl budget is worth your attention.
Google describes crawl budget as two parts:
- Crawl capacity limit: how much crawling your server can handle. Google says this "limits the total amount of time your server spends holding connections open for Google."
- Crawl demand: how much Google wants to crawl, driven by perceived inventory (Google tries to crawl most URLs it knows about unless you guide it), popularity, and staleness ("Our systems want to recrawl documents frequently enough to pick up any changes").
Large catalogs and programmatic SEO pages hit this status most often, because they create many similar URLs at once. Google's best practices in the same guide apply directly:
- Manage your URL inventory: tell Google which URLs to crawl and which not to.
- Consolidate duplicate content so crawling focuses on unique pages.
- Return 404 or 410 for permanently removed pages, and eliminate soft 404s.
- Keep sitemaps up to date.
- Avoid long redirect chains.
- Don't use robots.txt to temporarily shift crawl budget to other pages.
If you launch thousands of programmatic pages and most sit in this bucket, publishing fewer, better pages, then adding more as earlier ones get indexed, is usually more effective than trying to force the whole set through.
Server speed and crawl capacity
Server speed directly affects how much Google crawls. Google's crawl budget guide says that if a site slows down or returns errors, "the limit goes down and Google crawls less," and if it responds quickly and consistently, the limit goes up.
What to check:
- Crawl stats report. In Search Console, open Settings, then Crawl stats. Look at average response time, the breakdown by response code, and host status for availability problems.
- 5xx errors and timeouts. Even intermittent server errors make Google back off.
- Response time for HTML. Slow database queries on templated pages are a common bottleneck on large sites.
- Firewall and CDN rules. Bot protection that rate-limits or challenges Googlebot can look like an overloaded server.
Fixes that help: caching (Google's guide recommends supporting HTTP caching and 304 Not Modified responses), faster server response, lighter pages, and removing infinite URL spaces such as calendars or unbounded filters.
Internal links and sitemaps that signal priority
Internal links and sitemaps are how you tell Google which URLs matter most. A URL linked from frequently crawled pages gets found again and again; a URL that only exists in a sitemap competes on weaker terms.
Internal links
- Link new pages from your homepage, category hubs or a "latest" module that Google crawls often.
- Use normal
<a href>links with descriptive anchor text. - Keep important pages within a few clicks of the homepage.
- Fix orphan pages that appear in sitemaps but nowhere in your site's links.
Sitemaps
- List only canonical, indexable URLs you want crawled. Leave out redirects, noindexed pages and duplicates.
- Split large sites into multiple sitemaps (Google's limit is 50,000 URLs or 50MB uncompressed per file) and group them by section so you can see which sections lag.
- Set
<lastmod>to the date of the last significant change. Google says it useslastmodonly if it's "consistently and verifiably" accurate, and ignorespriorityandchangefreq(Build and submit a sitemap). - Submit sitemaps in Search Console and reference them in robots.txt.
How to get the pages crawled
To get discovered URLs crawled, improve the signals above and then ask Google to look. Work through these steps in order.
- Confirm the pages deserve crawling. Remove, noindex or consolidate URLs you don't actually want in search. Fewer low-value URLs means more attention for the ones you care about.
- Check server health in the Crawl stats report and fix errors or slow responses.
- Add internal links to the waiting pages from well-crawled pages.
- Update and resubmit your sitemap with accurate
lastmodvalues. - Request indexing for the most important URLs in the URL Inspection tool. There is a daily quota, so save it for pages that matter most. Google says requesting the same URL repeatedly "won't get it crawled any faster" (Ask Google to recrawl your URLs).
- Earn or build relevant links from pages Google already crawls. External links raise crawl demand.
- For larger lists, use a push service. This is the case FreshlyCited is built for: you can push undiscovered pages to Google after it checks which ones Google still can't see, and drip-feed large URL lists over several days instead of sending everything at once.
Google says crawling "can take anywhere from a few days to a few weeks." None of these steps forces a crawl, and no tool can make Google index anything. They make your pages a stronger candidate and put them in front of Google sooner. For more on each method, see how to get Google to crawl your site.
- Don't use robots.txt to temporarily shift crawl budget to other pages. Google says it "won't shift this newly available crawl budget to other pages unless Google is already hitting your site's crawl capacity limit."
- Don't use Google's Indexing API for general pages. It is documented only for job posting and livestream pages.
- Don't republish the same pages with new dates to make them look fresh.
Tracking progress until they're indexed
Track these URLs until they move out of "Discovered" and, ideally, into the index. A page can go from discovered to crawled and then land in "crawled - currently not indexed", which is a different problem with a different fix.
A simple tracking routine:
- Weekly: check the count for this reason in the Page indexing report. Remember the example list is capped at 1,000 URLs and isn't guaranteed to be complete.
- Per important URL: use URL Inspection to see whether a last crawl date has appeared.
- For large lists: run a bulk index check on the URLs you pushed and re-check after a few days, then weekly. Keep the results so you can see which sections are improving.
- Watch the transition: if pages are crawled but still not indexed, switch to fixing content value.
Indexing is also the first step for AI search. Google says pages must be indexed and eligible for a snippet to appear as supporting links in AI Overviews or AI Mode (AI features and your website). Being indexed makes a page eligible; quality and relevance still decide whether it's cited.
For the full crawl-to-index picture, read the Google indexing guide.
If you have a list of URLs stuck in this status and want to know which ones Google has seen since: Check your own links free. We check what Google sees, push only what it can't, and re-check so you have the answer in front of you.






