The Site Had Grown a Spine, But No Skeleton

"A website without a sitemap is not invisible. It is just making every crawler improvise."

At some point this site stopped being "a few pages and a blog" and turned into a mildly opinionated little ecosystem. There were static pages like /about, /work, and /bloq. There were 28 bloqs sitting in MDX. There were projects. There were bytes and blips coming out of Supabase. Everything existed. Everything linked to something. Everything looked fine if you were a human with a browser and a few spare minutes.

But machines do not browse with vibes. They operate on weighted signals.

That was the moment the lack of a sitemap started bothering me. Not in a dramatic, production-is-on-fire way. More in the quiet way, where something feels structurally incomplete and keeps staring at you from the corner of the room like an unplugged smoke detector. The site had routes. The site had content. The site even had opinions. What it did not have was a canonical declaration of its own topology.

I also had the more embarrassing thought: does a site of this scale even deserve a sitemap? This is not Wikipedia. This is not The Guardian. This is a personal website with a respectable number of pages and an absolutely unreasonable amount of emotional investment. There is a non-zero chance that adding a sitemap here is like giving a municipal zoning map to a chai stall.

I briefly told myself this was fine because "search engines are smart now," which is the SEO equivalent of saying "the garbage collector will probably handle it." Sometimes it does. Sometimes it also drags your weekend into the abyss. So I kept digging, partly because the question was real, partly because I had agents around, and partly because I am exactly the kind of developer who can turn "do I need a sitemap?" into a medium-length character study.

What a Sitemap Actually Is

"Most infrastructure is just a polite file that saves another system from guessing."

A sitemap is an XML document that tells crawlers what URLs exist, when they were last modified, how frequently they tend to change, and how important they are relative to the rest of the site. Not important in the cosmic sense. More in the "if you are short on time, maybe do not crawl my pagination dead-end before my homepage" sense.

That sounds small, and it is small. The whole thing is basically a structured list with manners. But the effect is larger than the file itself because a sitemap reduces ambiguity.

Without one, a crawler discovers your site the long way:

With one, you hand over a map:

That distinction matters. Crawlers are not reading your site like a curious person on a Sunday afternoon. They are building a model of it. They care about URL inventory, freshness signals, and how to allocate attention. A sitemap helps them do less inference.

If you open my sitemap now at , the XML is basically the site saying: here are the pages, and here is my best attempt at not sending you on a recursive walk through internal links with a rate limit.

It looks roughly like this:

Not glamorous. Very useful. Like documentation that’s actually accurate.

This is also where people start treating priority, changefreq, and lastmod like ceremonial herbs waved in Google's direction. I have done this too. I have stared at XML like it was a sacred document from the Temple of Organic Search and hoped a bot somewhere would nod respectfully. It is important to recover from this.

Those fields are signals, not commandments.

• lastmod says: this URL changed at this time.
• changefreq says: this class of content tends to change with this cadence.
• priority says: within this site, these URLs matter more than those URLs.

None of that guarantees indexing. None of that guarantees ranking. A sitemap is not a bribe. It is not a cheat code. It is a declaration of site structure and update semantics. If it works well, it is because it is honest.

That honesty matters even more on smaller sites. On a giant publication, the value of a sitemap is almost self-evident because nobody wants a crawler exploring the scenic depth-first route when the homepage is right there. On a smaller personal site, the question is subtler. The value is not scale alone. It is clarity.

A Short History of the Old Web Being Reasonable

"The old web ran on small agreements."

Sitemaps belong to an older tradition of the web, one I have become increasingly sentimental about without wanting to become unbearable at parties. The web used to have a lot of small contracts between publishers and readers, publishers and crawlers, one site and another site. Feeds. Links. Pingbacks. Plain HTML that did not need a venture-funded orchestration layer and a seven-slide go-to-market plan.

The sitemap sits in that lineage. It is a publisher saying: here is what exists on my server, here is what changes, here is what I think matters. It is almost quaint in its directness. No SDK. No dashboard with gradients. Just XML, standing there with a clipboard.

The family resemblance between HTML, CSS, XML, and XSLT becomes more obvious the longer you stare at this stuff. HTML, HyperText Markup Language, gave the early web its document structure and links. CSS, Cascading Style Sheets, arrived so presentation could live outside the document instead of being stapled directly onto it by desperate developers under fluorescent lighting. XML, Extensible Markup Language, had a different job: not "here is a web page," but "here is structured data with explicit meaning."

XSLT, Extensible Stylesheet Language Transformations, came from that same world. It is what you use when the source is XML and you want to transform it into something else, often a view that a browser can render more kindly. HTML publishes documents. CSS styles them. XML describes structured data. XSLT translates that structured data into a more human-facing shape. Which is a wonderfully dignified way of saying: it helps the browser stop making your machine file look like a clerical warning.

The funny thing is that this old contract matters again, maybe more than it did a few years ago. We are now building for systems that ingest websites programmatically all the time. Search crawlers still exist. So do archivers, feed readers, preview generators, retrieval pipelines, and increasingly agents that traverse the web at scale, often without prioritization.

If machines are going to consume your site, then machine legibility is not an edge case. It is part of the interface.

That is the philosophical reason a sitemap matters to me. It is not there because I think XML is sexy. It is there because a site should be able to state, in plain machine terms, what it contains. I had already been circling similar questions in my metadata and JSON-LD piece, but the sitemap made the same issue feel more infrastructural and less decorative.

The Engineering Question Was Not "Do I Need One?"

"The real question was: what exactly am I promising when I publish a sitemap?"

Once I stopped treating the sitemap as ceremonial SEO incense, the implementation became much more interesting.

This repo is not a single-source content system. It has a mixed topology:

That means the sitemap cannot be a static artifact I write once and forget. It has to be generated from the same surfaces the site itself depends on. In this repo, that logic lives in sitemap.xml/route.ts.

The working contract is simple:

I like this shape because it forces the right kind of argument. Every URL that enters the sitemap has to answer four questions:

1. Where is it?
2. When did it last change?
3. How often does this kind of page really change?
4. How much crawl attention should it receive relative to other pages here?

That is not just serialization. That is editorial policy disguised as infrastructure.

The route aggregates static pages like /, /about, /work, /bloq, /byte, and /blip. Then it adds bloq post URLs from the filesystem, project pages from local data, and individual byte and blip pages from Supabase. Only after that does it serialize everything into XML.

That order matters. Serialization is the easy part. The harder part is deciding what deserves representation.

And yes, before a sensible engineer asks whether this is overkill: maybe a little. But "overkill" is not a useless category if the work clarifies the system. A sitemap for a site this size is not mandatory. It is just one of those engineering choices where the implementation cost is low, the clarity gain is real, and the opportunity for me to overthink priority values is, unfortunately, abundant.

I gave the homepage a priority of 1, because if a crawler is going to understand the site at all, that is still the front door. Listing pages like /work and /bloq sit slightly lower because they are hubs. Individual bloqs and project pages sit below that because they are important, but not foundational. /byte and /blip get lower values because they are numerous, useful, and less central to the main architecture of the site.

This is the part where an engineer can disappear into a swamp of false precision. Is a blip 0.5 or 0.55? Does the homepage spiritually deserve 0.97? I am proud to report that I resisted this very serious temptation. Barely. A sitemap should communicate relative importance, not cosplay as a credit scoring system.

The same principle applies to changefreq. A bloq post is not "daily" just because I would enjoy that aspiration. A project page is not "hourly" because I once fixed a typo and felt powerful. These values only mean something if they describe actual update behavior. A sitemap is only as honest as the freshness semantics behind it.

That is also why the route uses:

That is six hours. Not because six hours emerged from a mountaintop vision quest, but because this is a personal site with a modest publishing cadence and no need to regenerate XML every time the wind changes direction. Revalidation is not a stray constant. It is the operational boundary between freshness and waste.

If I am honest, part of the justification here is also that this was a good dev experiment. The site already had multiple content sources, I had agents around, and the marginal cost of trying this was much lower than the marginal cost of continuing to wonder about it. Sometimes "I wanted to understand the system better" is a perfectly respectable reason to build a small piece of infrastructure. Sometimes it is also how you end up spending an evening discussing XML with a machine while your friends develop normal hobbies.

In the Age of Agents, Legibility Is Part of the Product

"Every system that reads your site is making a model of you."

I keep coming back to this because it is easy to undersell the point. A sitemap is not just for Googlebot. It is for any non-human system trying to build a coherent map of what your site contains.

That includes search crawlers, obviously. But it also includes systems that fetch metadata, assemble previews, archive content, and increasingly agentic tools that navigate a site by combining links, heuristics, and whatever partial world model they can construct before timing out and embarrassing themselves.

I say this with affection because I have also been that system. I have absolutely written code that wanders through route structures with the confidence of a man holding an upside-down map.

The relevance here is not hype. It is architecture. If the web is being read by machines more often, then explicit structure becomes more valuable, not less. A sitemap is one of the cleanest ways to expose that structure. It says: here is the address space, here is some freshness metadata, and here is my best attempt at telling you what is central.

That argument connects pretty directly to some of the other things I have been thinking about here, especially in One Content System, Three Doorways, where I was already wrestling with the fact that the same content can be exposed through different interfaces for different kinds of readers. A sitemap is just the most machine-forward version of that instinct.

That does not replace good internal linking. It does not replace clear information architecture. It does not absolve you of building sane routes. It is the complement, not the substitute.

In other words: the sitemap is not the site. It is the site explaining itself without needing a guided tour.

And Then I Let Humans Read It Too

"Machine-readable is the requirement. Human-readable is good manners."

This was the smaller refinement, and I want to keep it in its proper place.

Once existed, it was technically correct. Which is to say: useful for crawlers, mildly hostile for humans, and aesthetically similar to being handed a spreadsheet in a stairwell.

So I added an XSLT stylesheet reference to make the XML render more cleanly in a browser. It is an old trick, but a good one. The sitemap stays machine-first, the browser applies the transformation, and suddenly the file stops looking like a tax audit conducted by angle brackets.

In practice, it means the sitemap XML can stay machine-first while the browser gets a nicer projection of it. Same data, friendlier face. No JavaScript needed. Just a stylesheet instruction sitting at the top of the XML like a note saying, "please do not display me in my natural goblin form."

I like this because infrastructure should not have to be ugly to be respectable. If I am going to inspect a sitemap while debugging, or while explaining the site to someone else, I would prefer not to feel like I am reading the shipping manifest for a haunted warehouse.

And once I had done that for the sitemap, the same logic spilled into the RSS feed at . I had already written about adding RSS in I Added an RSS Feed, but after styling the sitemap it felt strange to leave the feed looking like it had been faxed in from 2004. So the RSS got the same courtesy treatment: still machine-readable, just less likely to make a human recoil.

Still, this is a courtesy layer. The sitemap earns its keep before the styling ever appears.

What I Actually Think the Sitemap Is Doing

"A good sitemap does not make a site important. It makes a site legible."

After building this, I do not think of a sitemap as an SEO accessory anymore. I think of it as a declaration of intent.

It says this site has a shape. These routes are real. These pages change with different cadences. Some surfaces are foundational, some are archival, some are supporting structures. If you are a machine trying to understand this place, here is the least chaotic version of the truth.

Does a site at this scale strictly need it? Probably not in the life-or-death sense. The site would continue existing. People would still find pages through links, feeds, and navigation. The sun would rise. My coffee would still go cold while I stare at route handlers.

But can I justify it? Yes, in three ways.

• It makes the site more legible to machines.
• It forces me to articulate the site's topology honestly.
• It was, undeniably, a good little dev experiment, the kind you get to run when agents are available and your curiosity has once again defeated your sense of proportion.

That feels increasingly relevant. We keep talking about design systems, metadata layers, structured content, retrieval pipelines, agent loops, and all the other modern machinery. Fair enough. But underneath all of that, a sitemap is still one of the cleanest acts of engineering courtesy on the web.

Also, and I say this with love, it is very funny that one of the most honest files in a modern codebase is an XML document. Not the analytics dashboard. Not the admin panel. Not the fancy CMS. Just a little machine-facing list of URLs quietly doing epistemology in the corner.

I spent longer than I should have on this because that is apparently my way of exploring now. But I am glad it exists. If machines are part of the audience, someone has to leave them a map. Better that the map be accurate, a little thoughtful, mildly stylish, linked to the rest of the site, and only slightly haunted.