Skip to content
Derar Sattouf

Sitemap lastmod From Git: Stop Guessing Your Post Dates

SEO & analytics

An SEO audit of this site had one line I could not argue with: almost every post in my sitemap reported its last change at one of two timestamps, a few hours apart. That was true. I had refreshed a lot of posts in two commits and stamped each batch with the time I pushed it.

Nothing about that is illegal, but it is useless. A lastmod that says "everything changed at 19:00" tells a crawler nothing about which pages are worth fetching again. Google's sitemap documentation says it uses the value only when it is consistently and verifiably accurate, and that it should reflect the last significant update to the page. Mine reflected my deploy schedule.

This post is what I changed, with the code. It is about a Next.js site with MDX posts, but the idea carries to any site where content lives in a repository.

Where the wrong dates came from

Every post has two frontmatter fields, publishedAt and updatedAt, and the sitemap reads them:

const modifiedAt = (post: Post) => new Date(post.metadata.updatedAt || post.metadata.publishedAt);

That code is fine. The problem is that a person types the dates, and I am not a reliable clock. After a bulk edit, the easy thing is to paste one timestamp into every file.

The second cause is more embarrassing, and it is the one worth learning from. My content check had a rule that failed the run when two posts shared a publish day. It sounded like discipline. In practice, after a week where I published several posts together, the only way to make the check pass was to move dates around until no two matched. A check that cannot be passed honestly gets passed dishonestly, and the result was dates that looked more natural and were less true.

So there were two things to fix: where the dates come from, and what the check asks for.

The rule

I wrote down what each date means before writing code:

  • publishedAt is the day the post first went public. It cannot be earlier than the day the file entered the repository.
  • updatedAt is the day the body of the post last changed. A commit that only touches frontmatter does not count. If the body has not changed since publication, the field is absent.

The second rule is the important one. If changing a date counted as a change, fixing the dates would itself make every post "updated today", and I would be back where I started.

Reading the dates from git

Git already knows both answers. The first commit that added the file gives the publish floor. Walking the history from newest to oldest and comparing the body at each step gives the last real edit.

const body = (source) => source.replace(/^---\r?\n[\s\S]*?\r?\n---\r?\n?/, "").trim();

const history = (file) =>
  git(["log", "--follow", "--format=%H%x09%as", "--name-only", "--", file]);
  // parsed into [{ hash, date, path }], newest first

export const gitDates = (file) => {
  const commits = history(file);
  if (!commits.length) return { firstCommitted: today(), bodyChanged: today() };

  const firstCommitted = commits[commits.length - 1].date;
  if (body(fs.readFileSync(file, "utf8")) !== bodyAt(commits[0])) {
    return { firstCommitted, bodyChanged: today() };
  }

  let current = bodyAt(commits[0]);
  for (let index = 0; index < commits.length - 1; index += 1) {
    const previous = bodyAt(commits[index + 1]);
    if (previous !== current) return { firstCommitted, bodyChanged: commits[index].date };
    current = previous;
  }
  return { firstCommitted, bodyChanged: firstCommitted };
};

A few details matter more than they look:

  • Strip frontmatter with a regex, not a YAML parser. Old revisions of a file can hold frontmatter that no longer parses. One of mine did, and the parser threw halfway through the history.
  • --follow plus --name-only. Posts get renamed. --follow keeps the history across the rename, and --name-only tells you which path to read at each commit.
  • Uncommitted renames have no history yet. After git mv, git log on the new path returns nothing until you commit. I read staged renames from git diff --cached --name-status -M and look up the old path instead. Without that, renaming a post would have reset its publish date to today.
  • An uncommitted body edit counts as today. The script compares the working file with the last commit first.

For most posts this runs two or three git show calls, because the loop stops at the first commit where the body differs. For about thirty files it takes a few seconds.

Why I write the dates into the files

The obvious design is to call git from sitemap.ts at build time. I did not, for one reason: my host builds from a shallow clone. With only the most recent commits available, "the first commit that touched this file" is whatever the oldest fetched commit happens to be, and every date quietly becomes wrong in production while looking right on my machine.

So the script writes the dates into frontmatter, and the build only reads frontmatter:

npm run sync:dates     # writes publishedAt / updatedAt that git history supports
npm run check:content  # fails when frontmatter and git disagree

The check also refuses to run its git comparison in a shallow clone, and says so, instead of reporting nonsense.

Changing what the check asks for

The date comparison is an error. If a file says it was updated on a day when its body did not change, the run fails and prints the command that fixes it.

The publish-day rule became a warning:

// A warning, never an error: failing here would push whoever runs the check
// to spread the dates out instead of telling the truth.
for (const [day, list] of byDay) {
  if (list.length > MAX_POSTS_PER_DAY) warn(day, `${list.length} posts were published on one day`);
}

Publishing many posts on one day is worth knowing about. It is not something a script should be able to block, because the only ways to satisfy it are to publish less or to lie, and the script cannot tell which one you chose.

The pages that are not posts

Tool pages had a different bug. Their sitemap entries used new Date(), so every deploy told crawlers that every tool page had just changed. I replaced that with one dated constant per toolkit, next to the guide text it describes, and the tools index takes the newest of them.

Case studies had the opposite problem: no updatedAt at all, so a page rewritten last week still reported its 2024 project date. The same script now covers them, with one difference. A case study's publishedAt is the date of the project, not of the page, so only its updatedAt is checked.

After the change, the sitemap has a spread of dates that I did not design: some case studies still say August 2025 because their text has not changed since, and every post I rewrote this week says this week.

What to look at in Search Console

None of this produces a visible result on day one. These are the places I check:

  • Sitemaps report. It shows when Google last read the file and how many URLs it discovered. If the last read date is old, resubmit the sitemap there. Google also reads the Sitemap: line in robots.txt.
  • URL Inspection. For a post you edited, compare the last crawl date with your lastmod. Over a few weeks, pages you really changed should be recrawled sooner than pages you did not. That is the only evidence that the value is being trusted.
  • The sitemap itself. Open /sitemap.xml and read it. If you see one timestamp repeated down the page, or the time of your last deploy, you have the bug I had.

Two things I stopped spending time on: priority and changefreq. Google's documentation says it ignores both. Mine are still in the file because the framework writes them, but I no longer tune them.

If you do this on your own site

  1. Decide what each date means, in one sentence each, and put it in the README.
  2. Take the dates from version control, and ignore commits that only change metadata.
  3. Write them into the content files so the build does not depend on git history.
  4. Make disagreement an error with a one-command fix.
  5. Keep any rule about publishing frequency as a warning.

The related pieces on this site are the Next.js technical SEO checklist, which covers the rest of sitemap.ts, and the structured data guide, because dateModified in your JSON-LD should come from the same field as lastmod.

Sources