SEO Skill

Sitemap URL extractor

Enter a sitemap URL or domain, follow nested sitemap indexes, then filter and download the extracted URLs with their sitemap metadata.

Enter a sitemap or domain

Use an exact sitemap URL, or enter a domain and we will check robots.txt and common sitemap locations.

The server fetches sitemap files only and returns up to 50,000 unique URLs. It does not crawl the pages listed inside them.

How to find a website’s sitemap

Most sites list their sitemap in robots.txt with a Sitemap: line, so that is the first place the extractor looks. If there is none, it tries the usual addresses, such as /sitemap.xml and /sitemap_index.xml. Some sites use other names, and some have no sitemap at all. When you know the address, enter it directly.

Extract URLs from sitemap files

The extractor reads XML sitemaps, sitemap indexes, gzip files, and text sitemaps. When you enter a domain, it checks robots.txt first and then common sitemap locations. It follows linked child sitemaps and shows any files it could not read.

CSV and JSON downloads retain each URL's source sitemap, nesting depth, last modified value, change frequency, priority, hreflang count, and image, video, or news entry counts. These are declarations in the sitemap. The tool does not request the page or confirm that its metadata is accurate.

Split large sitemap files around 10,000 URLs

Google accepts up to 50,000 URLs or 50 MB uncompressed in one sitemap. This tool flags individual files above 10,000 URL entries because many technical SEO teams use that as a practical working size. Smaller files can make failures, retries, and Search Console coverage groups easier to isolate. The 10,000 URL threshold is a convention, not a Google requirement or ranking factor.

Read Google's current sitemap size and format documentation for the protocol limits. Barry Adams also explains the 10,000 URL working size in his technical sitemap guidance.

Partial results stay visible

A blocked child sitemap, timeout, invalid XML file, or processing limit does not erase the URLs already extracted. The result is marked as partial and lists the affected sitemap. Do not treat a partial export as the complete URL inventory for the site.

Some sites block automated requests even when their sitemap works in a normal browser. Entering the exact sitemap URL may help when discovery fails, but it cannot bypass the target site's access controls.

The tool does not crawl extracted pages

HTTP status codes, canonicals, robots directives, and indexability need page requests. Run a technical SEO site crawl with the CLI when you need that evidence.

npm i -g seo && seo start

What you can do with the URL list

  • Count the pages a site says it has, by section, before an audit or a migration.
  • Build a redirect map from the old URL list before you move a site.
  • Compare a competitor’s sitemap month to month to see what they publish.
  • Find sections that are missing from your own sitemap, or listed twice.

The last modified dates show what a site says it changed and when. They are only as honest as the site that writes them.

Check the extracted URLs with your agent

A sitemap is a list of what a site claims. To see which of those pages work, ask your agent to check them with seo crawl, which can read the sitemap and check each URL’s status and redirects without reading every page. Ask:

Check every URL in https://example.com/sitemap.xml and list the ones that redirect or return an error

Or add the hosted MCP server to your agent and sign in with Google:

Run it in a terminal, then sign in with Google.

Or run it on your machine

Installs the CLI, signs you in and adds the skill to your agents. Site crawls and Bing data run this way, on your own machine. A crawl fetches many pages, and Bing blocks our hosted server.

Sitemap extractor questions