Oops! I Forgot My Website Sitemap (But Now It's Fixed!)

tl;dr: So, guess what? I totally spaced on adding a sitemap to my website. A sitemap helps search engines find all my awesome content, which is kind of important! This post will show you how I easily set it up with Astro, so you don't make the same mistake I did.

Astro has a built-in sitemap generator, so I just had to add the following to my astro.config.mjs:

export default defineConfig({
	site: "https://www.sanju.sh",
	integrations: [
        mdx(), 
        sitemap({
            changefreq: 'daily',
            priority: 0.7,
            lastmod: new Date('2024-06-01'),
        }), 
        tailwind()
    ],
	output: "server",
	adapter: cloudflare(),
});

I also added a robots.txt file to my website. I created a new file src/pages/robots.txt.ts with the following content:

import type { APIRoute } from 'astro';

const robotsTxt = `
User-agent: *
Allow: /

Sitemap: ${new URL('sitemap-index.xml', import.meta.env.SITE).href}
`.trim();

export const GET: APIRoute = () => {
  return new Response(robotsTxt, {
    headers: {
      'Content-Type': 'text/plain; charset=utf-8',
    },
  });
};

I also added the following to my package.json:

{
  "scripts": {
    "build": "astro build && astro sitemap"
  }
}

Now, whenever I run npm run build, Astro will generate a sitemap for me. I also submitted the sitemap to Google Search Console to help them index my content faster.

See you in the next ship!,

Need help with your website SEO? Let's chat!

P.S. follow me on Twitter i document my journey as a designer, developer and startup founder!