How to get Google to crawl your site and recrawl updated pages

Google says crawling can take anywhere from a few days to a few weeks, but you can make strong requests. Recrawls matter as much as first crawls. AI answers cite what's been crawled, and they lean toward pages that are kept current, so an update Google hasn't recrawled is an update search and AI answers can't use yet. FreshlyCited helps Google discover and index the pages, posts and backlinks you publish or update sooner, so they become eligible for Google Search, AI Overviews and AI Mode, then re-checks and shows you what Google has indexed. This guide covers how Google decides when to crawl, the difference between new and updated pages, each method for requesting a crawl, and how to verify the recrawl happened.

How Google decides when to crawl

Google decides when to crawl based on two things: how much crawling your server can handle (crawl capacity) and how much Google wants your URLs (crawl demand). Google explains both in its crawl budget guide.

Crawl capacity rises and falls with your server. If the site slows down or returns errors, "the limit goes down and Google crawls less." If it responds quickly and consistently, the limit goes up.

Crawl demand depends on three factors Google names:

• Perceived inventory: Google tries to crawl most URLs it knows about on your site unless you guide it.
• Popularity: "URLs that are more popular on the Internet tend to be crawled more often to keep them fresher in our systems."
• Staleness: "Our systems want to recrawl documents frequently enough to pick up any changes."

Google discovers new URLs by following links from pages it already knows and from sitemaps you submit (How Google Search works). Every method below works by improving one of these inputs: making URLs easier to discover, signaling that something changed, or raising how much Google wants the page.

New pages vs updated pages: crawl requests and recrawls

New pages and updated pages need different signals. A new URL needs to be discovered; an updated URL is already known, so Google needs a reason to come back sooner than it otherwise would.

Updated pages deserve more attention than they usually get. If you rewrite a pricing page or refresh a guide, the old version stays in Google's index until the next crawl. Search snippets and AI answers keep working from that old version in the meantime. If a new page is stuck before its first crawl, see discovered - currently not indexed.

Request a recrawl in Search Console

Request indexing in the URL Inspection tool is the direct way to ask Google to crawl a specific URL on a site you own. Google names it as the method for individual URLs in Ask Google to recrawl your URLs.

Steps

1. Open Search Console and select the property.
2. Paste the URL into the inspection bar at the top of the URL Inspection tool.
3. Click Test live URL to confirm the updated version is crawlable, indexable and renders correctly.
4. Click Request indexing.
5. Note the date and the current "Last crawl" value so you can compare later.

Rules worth knowing

• You must be an owner or full user of the property.
• There is a quota on individual URL submissions, so reserve it for important pages.
• "Requesting a recrawl multiple times for the same URL won't get it crawled any faster." Ask once per meaningful change.
• It only covers verified properties. For pages on other sites, such as where your backlinks live, you need a different route; see how to submit a URL to Google for all five methods.

Sitemaps and an accurate lastmod

A sitemap with an accurate lastmod is the scalable way to tell Google which pages changed and when. Google's sitemap documentation says it "uses the <lastmod> value if it's consistently and verifiably" accurate, and that the value "should reflect the date and time of the last significant update to the page."

What counts as a significant update

• Changes to the main body content, structured data or links on the page count.
• Updating a copyright date in the footer does not.

How to get it right

1. Generate lastmod from the content's real modification time in your CMS, not from the time the sitemap was built.
2. Only change lastmod when the page actually changed.
3. List only canonical, indexable URLs. Keep each file under 50,000 URLs or 50MB uncompressed.
4. Submit the sitemap in the Search Console Sitemaps report and add a Sitemap: line to robots.txt.
5. Don't tune priority or changefreq. Google says it ignores both.

Google deprecated its sitemap "ping" endpoint in 2023 and said it uses lastmod "as a signal for scheduling crawls to URLs that we previously discovered" (Google Search Central blog), so an honest lastmod is now the main sitemap-based recrawl signal. Bing asks for the same thing: Microsoft's July 2025 guidance says "Avoid setting lastmod to the time your sitemap was generated unless the content on that URL was actually updated" (Bing Webmaster Blog).

A sitemap where every URL shows today's date tells Google nothing. Google's own warning in the same post: if lastmod doesn't match reality, "eventually we're not going to believe you anymore when it comes to the last modified date of your pages."

Internal links from frequently crawled pages

Linking a page from pages Google crawls often is one of the most reliable ways to get it crawled, because Googlebot finds the link on its next routine visit. This works for both new pages and updated ones.

Steps

1. Find your most frequently crawled pages. Usually that's the homepage, main hubs and category pages, and popular articles. The Crawl stats report (Search Console, Settings) shows overall crawl activity.
2. Add contextual links from two or three of them to the new or updated page, with descriptive anchor text.
3. For updates, add or refresh a "recently updated" module on hub pages so changed URLs get a fresh link path.
4. Use plain <a href> links. Script-only navigation and nofollowed links are weaker or invisible paths.
5. Keep important pages within a few clicks of the homepage, and fix orphan pages that appear only in sitemaps.

External links help the same way and also raise popularity, one of Google's crawl demand factors. A relevant link from a page Google already has indexed is a strong discovery path.

Crawl budget: when it matters and how to protect it

Crawl budget is a real constraint mainly for large or fast-changing sites. Google says its guide is aimed at sites with roughly 1 million or more unique pages changing weekly, sites with 10,000 or more pages changing daily, and sites where a large share of URLs sit in "Discovered - currently not indexed." Google calls these figures rough estimates, not exact thresholds.

If your site is smaller, crawl budget is rarely the bottleneck. Discovery, internal links and page value usually are.

How to protect crawl budget (from Google's guide)

• Manage your URL inventory: tell Google which URLs to crawl and which not to.
• Consolidate duplicate content so crawling focuses on unique pages.
• Return 404 or 410 for permanently removed pages, and eliminate soft 404s.
• Keep sitemaps up to date.
• Avoid long redirect chains.
• Make pages fast to load, and support HTTP caching with 304 (Not Modified) responses.
• Don't use robots.txt to temporarily shift crawl budget to other pages.

Also watch for infinite URL spaces such as calendars, unbounded filters and session parameters. They quietly absorb crawling that your real pages need.

Pushing many URLs at once

When you have more URLs than Search Console's quota covers, or the URLs live on sites you don't own, a push service is the practical route. This is common for agencies refreshing content across many client sites, SaaS teams shipping batches of pages, and link builders who need Google to look at the pages their links are on.

How to approach a bulk recrawl

1. Check first. Find out which URLs Google already has in their current form, so you only spend effort on the rest.
2. Push only what changed or what's missing. Don't resubmit unchanged pages to look fresh.
3. Spread large batches out rather than sending thousands at once.
4. Re-check after a few days, then again later, and record which URLs moved.

FreshlyCited works this way: you can push updated pages for a recrawl, it checks first and pushes only what Google can't see, then re-checks. It can't force Google to crawl or index anything. No tool can. It puts the request in front of Google and shows you the result. For an updated page Google already has, choose Refresh anyway to send it: that uses 1 index credit when it's sent, with no credit back, because we can't see when Google re-crawls a page it already has. IndexNow notifications to Bing (for sites you verify) stay free.

Avoid one shortcut. Google's Indexing API is documented only for pages with JobPosting or BroadcastEvent-in-VideoObject markup (Using the Indexing API). Using it for other pages is outside its stated use.

Verify that Google recrawled the page

Verification is the step most people skip. A request doesn't mean Google came back, and a recrawl doesn't mean the new version is what's indexed.

For a page you own

1. Open URL Inspection and check Last crawl. A date after your request means Google came back.
2. Check that the Google-selected canonical is still your URL.
3. Use View crawled page to confirm Google stored the updated content.

For many pages, or pages you don't own

• Run an index check on the list and compare with your previous run.
• Search an exact new sentence from the updated page. If Google returns it, the new version is indexed. Absence isn't proof: Google says even site: results are "not always exhaustive."

Why this matters for AI answers: Google says its AI features "highlight content from our Search index" (Google's guide to optimizing for generative AI features), and a page must be indexed and snippet-eligible to appear as a supporting link (AI features and your website). Until Google recrawls your update, AI answers work from the old version. Studies point the same way: in Seer Interactive's 2026 study of 7,683 pages cited by ChatGPT, Gemini and Perplexity (March to June 2026), 75% had been updated within the past year (Seer Interactive). That's a correlation, not proof that updating causes citations, but it shows engines mostly cite maintained pages. Read more on why freshness matters for AI answers.

For the full crawl-and-index picture, see the Google indexing guide.

If you update pages regularly and want to know which updates Google has actually picked up: Check your own links free. We check what Google sees, push only what it can't, and re-check so you can see the result.