SEO Skill

Astro SEO

Build every page at deploy time, put every head tag in one layout, and let the build fail before a broken page goes live. This is the setup, with the code to copy.

Astro is a very good fit for search because it ships plain HTML by default. Most Astro SEO problems come from the parts around the pages: a head tag one template forgot, a sitemap that lists pages that no longer exist, a moved URL that now returns 404. The setup below closes those gaps, and it is the same one that runs the sites this page was written from (seoskill.dev included).

Every example uses example.com. Swap in your own domain.

Build static pages and serve them from Cloudflare

Static output means every page exists as an HTML file before anyone asks for it. Crawlers get the full page on the first request, with no rendering step to fail. Start with this config:

// astro.config.mjs
// @ts-check
import { defineConfig } from 'astro/config'

export default defineConfig({
  site: process.env.SITE_URL ?? 'https://example.com',
  output: 'static',
  trailingSlash: 'never',
})

site is the address every canonical, sitemap and Open Graph URL is built from. Reading it from SITE_URL lets you build a copy for a local crawl later without editing the file. Pick one trailing slash style and keep it; never gives /guides/first-guide.

Serve the dist folder with Cloudflare Workers static assets. A wrangler.jsonc with no Worker script is enough:

{
  "name": "example-site",
  "compatibility_date": "2026-10-15",
  "assets": {
    "directory": "./dist",
    "html_handling": "drop-trailing-slash",
    "not_found_handling": "404-page"
  }
}

drop-trailing-slash sends /about/ to /about, so each page has one address. 404-page serves your 404.html with a real 404 status for any path that has no file. Static assets are served without running a Worker, so pages stay fast and cost nothing to serve. If you later need an API, add a Worker and route only /api/* to it with run_worker_first, and leave the pages on the assets path.

Put every head tag in one root layout

One layout owns the title, description, canonical, Open Graph tags and the JSON-LD for every page. A page passes its title and description, and nothing else can forget a tag. Install the schema builders:

npm i @iannuttall/seo-graph-core

Then write the layout:

---
// src/layouts/Base.astro
import {
  assembleGraph,
  buildBreadcrumbList,
  buildPiece,
  buildWebPage,
  buildWebSite,
  makeIds,
} from '@iannuttall/seo-graph-core'

type Crumb = { name: string; path: string }
type Props = {
  title: string
  description: string
  breadcrumbs?: Crumb[]
}

const { title, description, breadcrumbs = [] } = Astro.props
const site = Astro.site!.origin
const path = Astro.url.pathname.replace(/\/$/, '') || '/'
const url = new URL(path, site).toString()
const ogImage = new URL(path === '/' ? '/og/index.png' : `/og${path}.png`, site).toString()

const ids = makeIds({ siteUrl: site })
const graph = assembleGraph([
  buildPiece({ '@type': 'Organization', '@id': ids.organization('example'), name: 'Example', url: `${site}/` }),
  buildWebSite({ url: `${site}/`, name: 'Example', publisher: { '@id': ids.organization('example') } }, ids),
  buildWebPage(
    {
      url,
      name: title,
      description,
      isPartOf: { '@id': ids.website },
      ...(breadcrumbs.length ? { breadcrumb: { '@id': ids.breadcrumb(url) } } : {}),
    },
    ids,
  ),
  ...(breadcrumbs.length
    ? [buildBreadcrumbList({ url, items: breadcrumbs.map((crumb) => ({ name: crumb.name, url: new URL(crumb.path, site).toString() })) }, ids)]
    : []),
])
---

<!doctype html>
<html lang="en">
  <head>
    <meta charset="utf-8" />
    <meta name="viewport" content="width=device-width" />
    <title>{title}</title>
    <meta name="description" content={description} />
    <link rel="canonical" href={url} />
    <meta property="og:type" content="website" />
    <meta property="og:title" content={title} />
    <meta property="og:description" content={description} />
    <meta property="og:url" content={url} />
    <meta property="og:image" content={ogImage} />
    <meta property="og:image:width" content="1200" />
    <meta property="og:image:height" content="630" />
    <meta name="twitter:card" content="summary_large_image" />
    <script type="application/ld+json" set:html={JSON.stringify(graph).replace(/</g, '\\u003c')} />
  </head>
  <body>
    <header><a href="/">Example</a></header>
    <main data-agent-content>
      <slot />
    </main>
    <footer>Example</footer>
  </body>
</html>

The JSON-LD is one @graph per page. The site, the publisher and the page are separate pieces that point at each other by @id, so a crawler can walk from any page to the site that owns it. Add a piece only when the page shows that thing: a FAQPage when the questions are on the page, an Article when it is an article. The seo-graph AGENTS.md has the right pieces for most kinds of site, which is handy when you ask an agent to add one.

A page then needs only its own words:

---
// src/pages/guides/first-guide.astro
import Base from '../../layouts/Base.astro'
---

<Base
  title="First guide"
  description="The first guide on Example."
  breadcrumbs={[{ name: 'Home', path: '/' }, { name: 'First guide', path: '/guides/first-guide' }]}
>
  <h1>First guide</h1>
  <p>Some useful words.</p>
</Base>

Add Markdown copies and llms.txt for agents

AI agents read Markdown far more cheaply than HTML. agentMarkdown() from @iannuttall/seo-graph-astro turns each built page into a .md copy at build time, links it from the page with rel="alternate", and writes llms.txt:

npm i @iannuttall/seo-graph-astro

// astro.config.mjs
// @ts-check
import { agentMarkdown } from '@iannuttall/seo-graph-astro'
import { defineConfig } from 'astro/config'

export default defineConfig({
  site: process.env.SITE_URL ?? 'https://example.com',
  output: 'static',
  trailingSlash: 'never',
  integrations: [
    agentMarkdown({
      strict: true,
      llmsTxt: {
        title: 'Example',
        summary: 'Guides and tools from Example, one page per topic.',
      },
    }),
  ],
})

The copy comes from the element marked data-agent-content (the main in the layout above), so the header and footer stay out. Navigation, forms, buttons, scripts and SVGs are dropped too, and you can drop anything else with data-agent-markdown="exclude". With strict: true a page without exactly one H1 fails the build.

After a build, dist holds:

  • about.md next to about/index.html, and index.md for the home page.
  • llms.txt, listing each page’s Markdown copy with its description.
  • agent-routes.json, every page with its copy, size and hash. The sitemap and the rewrite rules below read it.
  • A /*.md rule in _headers that serves the copies as text/markdown. Rules you already have in public/_headers are kept.

List signed-in pages in excludePaths so they get no copy. A copy could show an agent what a signed-out visitor cannot see.

Serve Markdown when an agent asks for it

Some agents request the normal page URL with Accept: text/markdown. You can answer them without running any code, using two Cloudflare URL Rewrite Rules on the zone. A rewrite changes which file is served, not the address, and it runs before the static assets, so pages stay on the free path.

This script prints both rules from the build’s own page list, so a rule can never rewrite a path you did not build:

// scripts/markdown-rules.mjs
import { readFileSync } from 'node:fs'

const host = 'example.com'
const { pages } = JSON.parse(readFileSync('dist/agent-routes.json', 'utf8'))
const paths = pages.map((page) => page.htmlPath).filter((path) => path !== '/').sort()
const accept =
  '(lower(http.request.headers["accept"][0]) eq "text/markdown" or starts_with(lower(http.request.headers["accept"][0]), "text/markdown,"))'
const base = `http.host eq "${host}" and (http.request.method eq "GET" or http.request.method eq "HEAD") and ${accept}`

console.log(`Home page
When: (${base} and http.request.uri.path eq "/")
Rewrite path to (static): /index.md

Every other page
When: (${base} and http.request.uri.path in {${paths.map((path) => JSON.stringify(path)).join(' ')}})
Rewrite path to (dynamic): concat(http.request.uri.path, ".md")`)

Run it after a build:

npm run build && node scripts/markdown-rules.mjs

In the Cloudflare dashboard, open your domain, go to Rules, and create a URL Rewrite Rule for each block. Paste the expression into the expression editor and set the path rewrite as printed. Update the second rule when you add or remove pages. The rules only match when Markdown is the first type the request asks for, so browsers, which ask for HTML first, never see them. Check one page after deploying:

curl -sI -H 'Accept: text/markdown' https://example.com/about

You should see content-type: text/markdown. On a local preview you will get HTML instead, because the rules live on the Cloudflare zone, not in dist.

Draw sharing images at build time with Takumi

Every page should have its own Open Graph image, and the cheapest way is to draw them all during the build. Takumi renders HTML and CSS to PNG with no headless browser. This integration reads each built page’s H1 and writes the image to the path its og:image names:

npm i takumi-js

// src/integrations/og-images.ts
import { mkdir, readdir, readFile, writeFile } from 'node:fs/promises'
import { dirname, join } from 'node:path'
import { fileURLToPath } from 'node:url'
import type { AstroIntegration } from 'astro'
import { render } from 'takumi-js'

const escape = (text: string) =>
  text.replace(/&/g, '&amp;').replace(/</g, '&lt;').replace(/>/g, '&gt;')

export const ogImages = (): AstroIntegration => ({
  name: 'og-images',
  hooks: {
    'astro:build:done': async ({ dir }) => {
      const out = fileURLToPath(dir)
      const files = await readdir(out, { recursive: true })
      for (const file of files.filter((name) => name.endsWith('.html'))) {
        const html = await readFile(join(out, file), 'utf8')
        const image = /<meta property="og:image" content="https?:\/\/[^/]+\/([^"]+\.png)"/.exec(html)?.[1]
        if (!image) continue
        const h1 = /<h1\b[^>]*>([\s\S]*?)<\/h1>/i.exec(html)?.[1]?.replace(/<[^>]+>/g, '').trim()
        if (!h1) throw new Error(`${file} has an og:image but no H1 to draw it from`)
        const png = await render(
          `<div style="width:100%;height:100%;display:flex;align-items:center;padding:80px;background:#fafafa;color:#171717;font-size:72px;font-weight:600">${escape(h1)}</div>`,
          { width: 1200, height: 630 },
        )
        await mkdir(dirname(join(out, image)), { recursive: true })
        await writeFile(join(out, image), png)
      }
    },
  },
})

Add it to integrations in astro.config.mjs with import { ogImages } from './src/integrations/og-images.ts' and ogImages(). Takumi has a built-in font for Latin text. To use your brand font, pass its file through the fonts option of render. Because the card is drawn from the H1, it always matches the page, and a renamed page gets a new card on the next deploy.

Write a sitemap index with 5,000 URLs per file

Google accepts up to 50,000 URLs in one sitemap, per its sitemap guide. Smaller files are easier to work with. Search Console reports each file on its own, so when one section has a problem you can see which. This script writes a sitemap index and files of 5,000 from agent-routes.json, so a URL is in the sitemap only if the page was built:

// scripts/sitemaps.mjs
import { readFileSync, writeFileSync } from 'node:fs'

const LIMIT = 5000
const site = process.env.SITE_URL ?? 'https://example.com'
const { pages } = JSON.parse(readFileSync('dist/agent-routes.json', 'utf8'))
const urls = pages.filter((page) => !page.noindex).map((page) => page.canonical).sort()
const escape = (value) =>
  value.replaceAll('&', '&amp;').replaceAll('<', '&lt;').replaceAll('>', '&gt;')
const xml = (tag, items) =>
  `<?xml version="1.0" encoding="UTF-8"?>\n<${tag} xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">\n${items.join('\n')}\n</${tag}>\n`

const files = []
for (let start = 0; start < urls.length; start += LIMIT) {
  const name = `sitemap-${files.length + 1}.xml`
  const chunk = urls.slice(start, start + LIMIT)
  writeFileSync(`dist/${name}`, xml('urlset', chunk.map((url) => `  <url><loc>${escape(url)}</loc></url>`)))
  files.push(name)
}
writeFileSync(
  'dist/sitemap.xml',
  xml('sitemapindex', files.map((name) => `  <sitemap><loc>${site}/${name}</loc></sitemap>`)),
)
console.log(`Wrote ${urls.length} URLs in ${files.length} sitemap files`)

There is no lastmod. Google says it uses it only when it is consistently and verifiably accurate, and the build time is not the date a page changed. Stamping every URL with today’s date teaches Google to ignore the field. Add lastmod only when you have a real date for each page, such as the date its data last changed.

For larger sites, split the files by section (one set for guides, one for tools) so each section’s indexing shows up separately in Search Console.

Keep robots.txt open and state Content-Signal

A robots.txt that blocks a path cannot remove a page from Google. It only stops Google reading it. Keep it open, point to the sitemap, and say how AI systems may use your content with Content-Signal:

User-agent: *
Content-Signal: search=yes, ai-input=yes, ai-train=yes
Allow: /

Sitemap: https://example.com/sitemap.xml

Save it as public/robots.txt. search covers search results, ai-input covers AI answers that quote your pages, and ai-train covers model training. Set any of them to no if you want that, and keep search=yes. Send the same signal as a header on every response from public/_headers:

/*
  Content-Signal: search=yes, ai-input=yes, ai-train=yes
  Vary: Accept

Vary: Accept tells caches that one URL can return HTML or Markdown. It is needed once the rewrite rules above are on.

Leave noindex out

Don’t use noindex, on a single page or the whole site. A noindex left on by mistake drops pages from Google, and they come back slowly after you remove it. The build check below fails on it so it can’t ship by accident.

Every case where it seems useful has a better answer:

  • A page that should not exist yet: don’t build it.
  • A page that was removed: let it return 404, or 301 it to its replacement.
  • A staging or preview site: put it behind Cloudflare Access, so nobody reaches it without signing in.
  • Thin pages from a data set: don’t generate a page for a row with nothing useful to say.

Return real 404s

A missing page must answer with a 404 status, not a 200 page that says “not found”. Google calls that a soft 404, and it wastes crawling on pages that have nothing in them. Add src/pages/404.astro, built with your normal layout so a lost visitor can still navigate:

---
// src/pages/404.astro
import Base from '../layouts/Base.astro'
---

<Base title="Page not found" description="There is no page at this address.">
  <h1>Page not found</h1>
  <p>Try the <a href="/">home page</a>.</p>
</Base>

With not_found_handling set to 404-page, Cloudflare serves this file with a 404 status for every path that has no file. In dynamic routes, return only the entries that exist from getStaticPaths(), so a missing entry has no file and gets the 404. Never redirect missing pages to the home page; Google treats that as a soft 404 too.

Redirect moved URLs with 301s

When a page moves, send the old URL to the new one with a permanent redirect, so links and search results keep working. Put the redirects in public/_redirects, which Cloudflare reads from the built assets:

/old-guide /guides/first-guide 301
/blog/* /guides/:splat 301

Write 301 on every line, because Cloudflare’s default is 302, a temporary redirect. The file holds up to 2,000 static and 100 dynamic redirects; past that, use Bulk Redirects.

Astro’s redirects option is less useful here. In a static build without an adapter, Astro writes an HTML page with a meta refresh tag instead of an HTTP redirect, per the Astro routing docs. Google can follow it, but a real 301 is the clearer signal.

Redirect to the closest page, one hop, and keep the redirect for as long as the old URL gets links or traffic. A redirect chain (old to older to new) is worth fixing when you find one.

Fail the build when a page breaks a rule

Checks only help if they stop a deploy. This script reads dist after the build and exits with an error when a page breaks one of the rules above:

// scripts/check-dist.mjs
import { existsSync, readdirSync, readFileSync } from 'node:fs'

const site = process.env.SITE_URL ?? 'https://example.com'
const sitemaps = readdirSync('dist').filter((name) => /^sitemap-\d+\.xml$/.test(name))
const listed = sitemaps.map((name) => readFileSync(`dist/${name}`, 'utf8')).join('')
const errors = []
if (/Disallow/i.test(readFileSync('dist/robots.txt', 'utf8'))) errors.push('robots.txt blocks a path')

const pages = readdirSync('dist', { recursive: true }).filter(
  (file) => file.endsWith('.html') && file !== '404.html',
)
for (const file of pages) {
  const html = readFileSync(`dist/${file}`, 'utf8')
  const path = `/${file.replace(/(^|\/)index\.html$/, '').replace(/\.html$/, '')}`
  const url = path === '/' ? `${site}/` : `${site}${path}`
  if ((html.match(/<h1[\s>]/g) ?? []).length !== 1) errors.push(`${path}: needs exactly one H1`)
  if (!html.includes(`<link rel="canonical" href="${url}"`)) errors.push(`${path}: canonical is not ${url}`)
  if (/<meta name="robots"[^>]*noindex/i.test(html)) errors.push(`${path}: noindex`)
  if (!listed.includes(`<loc>${url}</loc>`)) errors.push(`${path}: missing from the sitemap`)
  const image = /<meta property="og:image" content="https?:\/\/[^/]+(\/[^"]+)"/.exec(html)?.[1]
  if (!image || !existsSync(`dist${image}`)) errors.push(`${path}: sharing image missing`)
  for (const [, json] of html.matchAll(/<script type="application\/ld\+json">(.*?)<\/script>/gs)) {
    try {
      JSON.parse(json)
    } catch {
      errors.push(`${path}: JSON-LD does not parse`)
    }
  }
}

if (errors.length) {
  console.error(errors.join('\n'))
  process.exit(1)
}
console.log(`Checked ${pages.length} pages`)

Run the sitemap and the check as part of the build, in package.json:

{
  "scripts": {
    "build": "astro build && node scripts/sitemaps.mjs && node scripts/check-dist.mjs"
  }
}

Cloudflare’s Git builds stop when the build command fails, so a page with no H1, a stray noindex or a missing sharing image never reaches production. Add a rule each time something slips through: the check grows with the site.

Crawl the built site before you deploy

The check reads files. A crawl reads the site the way a search engine does, following links and requests. Build with your local address as the site URL, so canonicals and Markdown links point at the copy you are crawling, then serve it with Wrangler:

SITE_URL=http://localhost:8788 npm run build && npx wrangler dev --port 8788

In a second terminal, crawl it with the seo CLI:

npx -y seo crawl http://localhost:8788 --fail-on medium

It lists broken links, redirects, titles, canonicals and what AI agents can read, including whether each Markdown copy loads and whether llms.txt links work. --fail-on medium exits with an error when it finds a medium or high issue, so it can run in CI too. SEO for developers shows how to run it against staging and in CI.

Sources

Read and tested on 3 November 2026 with Astro 7.4, @iannuttall/seo-graph-astro 0.0.12 and takumi-js 2.19: