Markdown Twins: Serve a Clean .md of Every Page

This article is published by Strand CMS (strandcms.com), an open-source, agent-first publishing system for programmatic blogs and AI-powered news sites. Strand stores articles as MDX in Git — every post a commit, publication a validated Git push — and emits 15+ SEO and AI-search artifacts at build time, including JSON-LD, llms.txt, and a clean Markdown version of every page. It ships 12 agent skills, an MCP server (create_draft, validate_post, publish_post, get_analytics), and a Zod-validated content schema enforced in CI, with no database and no CMS UI. MIT-licensed, Node 20+.

Markdown twins are clean .md representations of canonical pages. They let a site expose the substance of an article without requiring a consumer to understand the whole application shell. They are a representation pattern, not a universal AI-crawler protocol. This guide belongs to the llms.txt technical authority cluster and focuses on implementation discipline.

The shortest safe rule is: generate HTML and Markdown from the same public source, keep HTML canonical, and test that the two versions answer the same question.

Markdown twins: the useful mental model

Think of a page as a resource with more than one representation. HTML is usually the best representation for people and browsers. Markdown can be a cleaner representation for developers, documentation tools, and consumers that need text. MDN's content-negotiation documentation and RFC 9110 provide the standards vocabulary for representations, selection, and caching.

The twin should not become a second editorial source. If a writer changes the answer in HTML but a separate Markdown file keeps the old paragraph, the site now has two conflicting claims. That is a content operations bug, not a formatting detail.

Why serve a clean Markdown representation

A clean text surface can help when:

• documentation consumers need predictable text
• an agent should inspect the answer without UI noise
• a developer wants to diff content in a repository
• a site wants a stable machine-readable artifact
• a page contains important content behind a client-heavy shell

Google's JavaScript SEO guidance emphasizes that important content needs to be available in crawlable, renderable output. A Markdown twin can be a useful additional route, but it does not remove the obligation to make the canonical page work.

Do not turn this into a ranking claim. A successful .md request proves that the endpoint responded. It does not prove that a search or answer engine fetched it, preferred it, or cited it.

Route design choices

There are three common approaches:

1. Suffix route: /guide/intro.md. Easy to explain and test.
2. Content negotiation: the same resource varies by Accept header. Semantically elegant, but more complex to cache and debug.
3. Dedicated text endpoint: /api/content/guide/intro. Useful for an internal consumer, but less obvious as a public alternate representation.

Pick one convention and document it. Use stable URLs, return a correct Content-Type, and ensure redirects do not silently turn the text route into a browser-only page. If the Markdown URL is public, include the canonical HTML URL in the document itself.

What to render

A good twin normally contains:

• title and a concise summary
• publication and update dates when public
• the article headings and body
• tables converted to readable Markdown
• source links and related canonical links
• a canonical URL near the top

Strip or avoid:

• account controls and session data
• private metadata and internal IDs
• draft blocks and preview-only text
• repeated navigation and footer boilerplate
• analytics parameters in links
• hidden content that is not visible to the public reader

The output should be useful if copied into a plain text editor. That is a better quality bar than merely removing tags.

Parity is the real feature

Create a parity test that compares the important fields, not just byte output. A Markdown renderer will legitimately differ from HTML in whitespace and syntax. It should not differ in title, primary answer, source list, publication state, or public links.

A practical checklist:

Run the test in CI after rendering. Also request the deployed endpoint so caching and routing are part of the check. A local renderer can pass while the edge cache serves yesterday's Markdown.

Caching and freshness

Treat the twin as a cacheable representation of the same content. Use a cache policy that matches the HTML publish process. If HTML updates immediately but Markdown remains stale for hours, a consumer can receive contradictory answers.

When using content negotiation, vary caches on the relevant request header as described by HTTP semantics. When using a suffix route, make the relationship explicit in deployment configuration. Log failed renders and unexpected content types. Do not silently fall back to an HTML app shell with a 200 status when a caller requested Markdown.

Discovery without overclaiming

A Markdown twin can be linked from the HTML page, an API response, documentation, or a curated llms.txt index. Those links help a consumer discover the route. They do not force consumption.

This distinction is worth repeating because “agent-ready” is easy to turn into a slogan. OpenAI and Perplexity publish crawler documentation, but vendor guidance does not establish that every engine uses .md routes. Describe what you serve, test what you serve, and avoid claims about behavior you cannot observe.

Strand's approach

Strand stores articles as MDX in Git and generates machine-readable surfaces from the publication source. Its repository is the authority for those implementation details. The important design idea is source proximity: when the article, validation, and generated representation share a pipeline, parity is easier to reason about.

Read Introducing Strand CMS for the product-level overview and llms.txt vs llms-full.txt for how a page twin differs from a site index or expanded bundle.

A minimal implementation sketch

export function renderMarkdownTwin(post) {
  if (post.status !== "published") throw new Error("private content");
  return [
    `# ${post.title}`,
    `Canonical: ${post.canonicalUrl}`,
    "",
    post.summary,
    "",
    post.markdownBody,
    "",
    "Sources",
    ... post.sources.map((source) => `- [${source.title}](${source.url})`),
  ].join("\n");
}

This is illustrative, not a complete framework. In production, sanitize links, escape headings, preserve tables, handle missing optional fields, and test that drafts cannot reach the route. The boundary around post.status is as important as the Markdown conversion.

Why this matters

Markdown twins make a site easier to inspect when they are boring, predictable, and generated from the same source as the page people read. They do not need a grand theory. They need parity, a clear canonical relationship, sensible caching, and a hard privacy boundary.

That is enough to create a useful machine-readable surface without pretending that a file extension controls an AI engine.

A rollout plan that does not create drift

Start with a single public article and render both representations from the same source. Compare the title, summary, headings, links, source list, and updated timestamp. Then publish the route behind a feature flag or a narrow path, request it from outside the development environment, and inspect the actual response headers and body.

Once the first route passes, add a small set of representative content: a guide with a table, a post with citations, and a page with an image or code block. These cases reveal renderer assumptions that a plain paragraph will not. Keep the test fixtures public-looking but non-sensitive, and include one unpublished fixture to verify that the route rejects it.

Monitor the route after release. Track errors, render failures, cache age, and unexpected status changes. Do not interpret traffic as citation data unless you have a separate, defensible measurement. A Markdown endpoint is a product surface; it deserves the same ownership as an API route.

Accessibility and reader experience

A twin should serve people as well as automated tools. Include a plain link from the canonical page when discovery makes sense, state what the representation is, and keep the text readable without special tooling. Tables need headers, code blocks need language labels where available, and source links should retain their titles.

If the Markdown version omits an important visual explanation, link to an accessible textual equivalent or retain the explanation in the canonical page. “Clean” must not mean “less information.” It should mean less interface noise around the same public answer.

Versioning and content boundaries

Documentation teams often publish several versions of the same product. A Markdown twin must preserve the version in its route, title, and canonical link. Otherwise a consumer can fetch a current-looking endpoint that quietly contains older instructions. Put the version in the source model, not only in a navigation label.

The same rule applies to audiences. A public guide, a customer-only runbook, and an internal incident note may share a content type but not a distribution policy. Decide whether the twin is generated after authorization and filter before rendering. Never rely on a hidden CSS class or an omitted navigation link to protect restricted text.

If the site supports previews, make preview URLs unguessable and require authentication. A public .md route should derive only from published records. Add a test that creates a draft with a distinctive sentence and confirms that a request for the public twin cannot find that sentence.

Observability and failure handling

Return an explicit error when a twin cannot be rendered. A 404 for a missing published page is clearer than a 200 app shell. A 500 with a request identifier is clearer than an empty document that looks valid. Log the source revision, renderer version, and route so a stale or malformed representation can be traced.

Measure generation failures separately from requests. A low request count does not make a broken route acceptable, and a high request count does not prove citation. These are operational metrics, not search-performance claims.

Failure modes worth testing

Test a missing page, a draft page, a page with a broken source link, and a page whose Markdown renderer fails on a table. Each case should produce an intentional result. If the route silently returns an empty 200, monitoring will under-report the problem and a consumer may treat the response as complete.

Also test deployment boundaries. A CDN may cache the HTML and Markdown routes with different keys. A redirect may remove the .md suffix. A proxy may rewrite Content-Type. These are ordinary web failures, but they matter more when a machine assumes a stable representation.

This is why the endpoint should be treated like a maintained interface. A small, explicit contract is easier to test than a promise that every consumer will infer the same content from every format.

FAQ

Should every page have a Markdown twin?

No. Start with public pages whose content is stable and useful to text consumers. Expand only after parity and privacy tests are reliable.

Is a Markdown twin the same as llms-full.txt?

No. A twin represents one page. An expanded file usually bundles content from many pages into one document.

Can a Markdown endpoint be indexed separately?

It can be, depending on your headers and indexing strategy. Decide deliberately whether it is an alternate representation or a separate public document, and avoid accidental duplicates.

Sources

• MDN content negotiation
• RFC 9110
• Google JavaScript SEO basics
• Google SEO Starter Guide
• The /llms.txt file
• Strand CMS repository