--- title: "Crawled, currently not indexed: check every page | SEO Skill" description: "Find every page Google crawled and left out of its index, with Google's own reason, a live check of each page, and what to look at before you change it." canonical: "https://seoskill.dev/features/crawled-currently-not-indexed" language: "en" --- # Crawled, currently not indexed Find every page on your site that Google crawled and left out of its index. Your agent reads Google's own reason for each one and checks the live page, so you can tell the pages worth fixing from the ones to leave alone. Run it in a terminal, then sign in with Google. Or run it on your machine Installs the CLI, signs you in and adds the skill to your agents. Site crawls and Bing data run this way, on your own machine. A crawl fetches many pages, and Bing blocks our hosted server. ## What crawled, currently not indexed means Crawled, currently not indexed is a reason in Search Console's Page indexing report. Google fetched the page and chose not to add it to its index. Google's [Page indexing report help](https://support.google.com/webmasters/answer/7440203) says Google does not index every page on the web, so this reason "doesn't necessarily indicate an issue that needs attention". It gives no further detail, in the report or in URL Inspection. So the reason alone tells you very little. The useful work is finding which affected pages you actually want in Google, then reading each one for something Google would skip. ## Find every crawled, not indexed page on your site Search Console's report shows up to 1,000 example URLs for each reason, and not necessarily all of them. SEO Skill asks Google's URL Inspection about each page it knows from your Search Console and your sitemap, and sorts them by reason. 1. Connect the hosted MCP server and sign in with Google, or run `npm i -g seo` then `seo start`. Both are free. The site has to be a Search Console property your Google account can read. 2. Ask your agent: ``` Which pages on example.com are crawled but not indexed, and which of them matter? ``` Or run it in a terminal: ``` seo index --site example.com ``` It checks pages in a fixed order: pages that lost their clicks, then pages by clicks and impressions, then sitemap pages, then the rest. Each run checks 100. On your own machine you can check up to the quota Google has left today, which is 2,000 checks a day for each property: ``` seo index --site example.com --max-urls 500 ``` The result says how many pages it checked out of how many it knows. A page it did not reach has no result, so a short run is never an all-clear. ## Read the evidence for each page before you change it A page with this reason that still loads comes back labelled check, which means a person should look. One that now returns 404 while your sitemap or your own pages still point to it is labelled fix. Next to Google's reason you get what the page does now: | Evidence | What it tells you | | --- | --- | | Live status and redirect | Whether the page still answers 200, or has moved or broken since Google crawled it. | | In your sitemap | Whether you told Google you want this page indexed. | | Canonical, yours and Google's | Whether the page points Google at another URL, or Google picked a different one. | | Pages linking to it | The pages Google saw linking to it, as of its last crawl. | | Last crawl | How old Google's view of the page is. | SEO Skill's labels come from simple rules over this evidence. Google does not set them, so treat check as a question for you to answer. ## Pages you can leave out of Google Plenty of crawled, not indexed pages never needed to be search results: feeds, tag and filter pages, internal search results, deep pagination and plain text files. If a page is not in your sitemap and nothing you care about links to it, leaving it alone is usually the right call. Start with the pages your sitemap lists. Those are the ones you have told Google you want indexed, so they are where this reason costs you. ## What to check on a page you want indexed Give your agent the job for the whole list at once: ``` For each crawled, not indexed page on example.com that is in the sitemap, read the page, compare it with our other pages on the same topic, and tell me whether to improve it, merge it or leave it out. ``` Check three things on each page: - Read what the page says. On your machine, `seo page --url https://example.com/guides/composting --detail full` returns the page's main text as Markdown, as your server sends it. A template with a few fields filled in reads very differently from a page someone wrote. - Look for another page on your site that answers the same search. Two pages that cover the same thing usually read better merged into one, whatever Google does next. - See which of your pages link to it. The result lists the pages Google saw linking there, and `seo internal-links --url https://example.com/guides/composting` crawls your site and returns up to eight pages that should link to it, each with a sentence the link fits in. None of this makes Google index the page. A better page is a better candidate, and Google still decides. ## Check the page again after you change it For one page you just changed, check that URL on its own: ``` seo page --url https://example.com/guides/composting ``` URL Inspection reports Google's last known state, so it lags behind your change until Google crawls the page again. Compare the last crawl date with the day you made the change. `seo` cannot submit URLs to Google. To ask for a fresh crawl, use Request indexing in Search Console's [URL Inspection tool](https://support.google.com/webmasters/answer/9012289). To see whether Google search shows the page, ask for a live index check. It searches Google for `site:` plus the URL and for an exact phrase from the page, on desktop and mobile, and answers indexed, not found or unknown. It needs a DataForSEO key and costs about 2 to 4 cents a page: ``` seo research index_check --target https://example.com/guides/composting ``` That is Google search, not Search Console, so not found is evidence, not proof. ## One crawled, not indexed URL out of 272 on seoskill.dev We ran `seo index` on this site on 11 November 2026. Google had a current answer for all 272 URLs SEO Skill knows: 185 indexed, 79 redirects, 4 noindex pages, 3 unknown to Google and 1 crawled, currently not indexed. That one URL is `/llms.txt`, a plain text file for AI agents that our sitemap does not list. The rules label it check, like any page with this reason that still loads. We looked and left it alone: it was never meant to be a search result. ## Crawled or discovered, currently not indexed Crawled means Google fetched the page, so URL Inspection has a last crawl date. [Discovered, currently not indexed](https://seoskill.dev/features/discovered-currently-not-indexed) means Google knows the URL and has not fetched it yet, so the last crawl date is empty. They point at different checks: the page itself for crawled, your server and your links for discovered. ## Hear when a page drops out of Google's index The [weekly site audit](https://seoskill.dev/docs/weekly-site-audit) ($50 a month per site) runs the index checks for you on our servers, using each property's full 2,000 checks a day. Your top pages are checked daily. When a page drops out of the index you get one email, grouped by Google's reason, and a top page's drop is only sent after a second check agrees. [Alerts](https://seoskill.dev/docs/alerts) covers each email. ## Read the docs [ ### seo index Google's indexing reason for every known page, each checked live and labelled. seo index docs](https://seoskill.dev/docs/indexing)[ ### seo page One URL: what it serves now, its Google index status and its queries. seo page docs](https://seoskill.dev/docs/page)[ ### Discovered, currently not indexed Pages Google knows about but has not crawled yet. Discovered, not indexed](https://seoskill.dev/features/discovered-currently-not-indexed)[ ### Website not showing up on Google Every indexing reason across your site, and what to check for each. Website not on Google](https://seoskill.dev/features/website-not-showing-up-on-google)[ ### seo internal-links Find the pages on your site that should link to one page. seo internal-links docs](https://seoskill.dev/docs/internal-links)[ ### Weekly site audit Daily Google index checks and an email when a page drops out. Weekly site audit docs](https://seoskill.dev/docs/weekly-site-audit)