Read Your Competitor's Sitemap to Reverse-Engineer Their Content Strategy

Most people study a competitor's homepage. It shows what they want you to see. Their sitemap.xml shows what they actually built: every service page, blog post, category and landing page they want search engines to find, in one file. Read it well and you can see how they've structured their site, and where they haven't published anything yet.

What a sitemap is (and why it's public)

A sitemap is a file where a website lists the URLs it wants search engines to crawl. It's public by design, so search engines can read it. Reading one is like reading a public index, not breaking into anything. Reading it manually in a browser is fine. If you automate requests, be polite: fetch it once, don't hammer the server, and follow the site's terms.

How to find a competitor's sitemap

• Try the usual addresses: /sitemap.xml, /sitemap_index.xml and /sitemap-index.xml on their domain.
• Check robots.txt: open /robots.txt and look for a line starting with Sitemap:. That line can point to a sitemap at any address, and there can be several.
• Expect an index file: larger sites often publish a sitemap index that lists several smaller sitemaps (for example one for blog posts, one for products).

Some sites don't publish a sitemap at all, and some list only part of their pages. Treat what you find as a partial picture.

What a sitemap can reveal

• Content architecture — how they group pages: /services/, /blog/, /resources/, /locations/.
• Topics they invest in — many pages in one section usually means it's a priority.
• Service and landing pages — which offers they promote with dedicated pages.
• Blog scale and themes — how many posts, and which topics recur in the URLs.
• Location or audience pages — signs of local SEO or segment targeting.
• Update activity — the lastmod dates, with a caveat covered below.

The workflow

• Find the sitemap and any child sitemaps.
• Extract every URL into a spreadsheet.
• Group the URLs by their first path segment (/blog/, /services/) and then by topic keywords in the slug.
• Compare the result with your own sitemap, section by section.
• Decide which gaps are worth filling, and in what order.

Find content gaps

Once you have your competitor's URLs grouped, look for these patterns:

• Topics they cover that you don't — a cluster of pages about a subject you've never written on.
• Sections they have that you lack — for example a /case-studies/ or /guides/ section.
• Service pages you're missing — offers they explain in depth while you mention them in passing.
• Depth differences — they have twelve pages where you have one.
• Topics neither of you covers — the real opening.

A gap only tells you a topic exists. Whether it's worth covering depends on your audience and whether people search for it, which you check separately (for example in Google Search Console or a keyword tool). Don't copy their pages. Write your own, based on what your customers need.

Read the sitemap with healthy skepticism

• Not every page is listed. A sitemap is what the site chooses to submit. Pages can be missing, and some sitemaps are auto-generated with gaps.
• Listed isn't the same as ranking. A page in the sitemap may be thin, unindexed or unimportant. Google says submitting a sitemap is a hint and doesn't guarantee crawling.
• Ignore priority and changefreq. Google says it ignores the <priority> and <changefreq> tags, so don't read strategy into them.
• Be careful with lastmod. Google only uses lastmod when it's consistently and verifiably accurate. Many sites set it to the same date for every page, so a recent date doesn't prove real updates.
• Check for auto-generated pages. Tag pages, filters and pagination can inflate a URL count without being real content.

A small script to map a sitemap

You can do the extract-and-group step in a spreadsheet, but this short Python script does it for you. It reads a sitemap (following sitemap index files) and counts URLs by top-level section. Run it once against a sitemap you're entitled to read, and respect the site's terms.

import sys, urllib.request
import xml.etree.ElementTree as ET
from collections import Counter
from urllib.parse import urlparse

def fetch(url):
    req = urllib.request.Request(url, headers={"User-Agent": "sitemap-research/1.0"})
    with urllib.request.urlopen(req, timeout=20) as r:
        return ET.fromstring(r.read())

def urls(sitemap_url):
    root = fetch(sitemap_url)
    ns = "{http://www.sitemaps.org/schemas/sitemap/0.9}"
    if root.tag == ns + "sitemapindex":          # index file: follow child sitemaps
        for loc in root.iter(ns + "loc"):
            yield from urls(loc.text.strip())
    else:
        for loc in root.iter(ns + "loc"):
            yield loc.text.strip()

all_urls = list(urls(sys.argv[1]))
groups = Counter()
for u in all_urls:
    parts = [p for p in urlparse(u).path.split("/") if p]
    groups[parts[0] if parts else "(home)"] += 1

print(f"{len(all_urls)} URLs")
for section, n in groups.most_common():
    print(f"{n:4d}  /{section}/")

Run it as python sitemap_map.py https://example.com/sitemap.xml. It prints the total URL count and how many URLs sit under each section, which is a quick picture of where a site puts its effort.

Use AI to structure the results

Once you have the URL list, an AI assistant can group it into a content architecture faster than doing it by hand. Give it clear instructions and check the output, because it can misread a slug or over-generalize.

You are an SEO content strategist. Below is a list of URLs from a competitor's sitemap.

My business: [describe it in one sentence]

1. Group the URLs into sections and topic clusters, based only on the URL paths.
2. For each cluster, say how many URLs it has.
3. List the site's main service or landing pages.
4. Compare with my own site sections below and list topics the competitor covers that I don't.
5. Do not guess traffic, rankings, or performance. Work only from the URLs. Flag anything you are unsure about.

Competitor URLs:
[paste URLs here]

My site sections:
[paste my section list here]

Where this fits with your own site

Run the same exercise on your own sitemap first. It shows how your site looks to a search engine, and it often reveals orphaned sections or thin areas you'd want to fix before chasing a competitor. Day 4 of this series covers what to focus on for search visibility, and Day 6 covers using competitor reviews for market research.

Use it responsibly

• Study structure and topics, not their copy. Never copy a competitor's pages or text.
• Reading a public sitemap is normal research. Aggressive automated crawling of a site is not, so keep any requests light and respect its robots.txt and terms.
• Don't treat a competitor's structure as proof of what works. They may have made mistakes too.