MTMoueen Togarvi5 min read

llms.txt Implementation: A Useful Guide Without SEO Promises

Quick overview

Updated

Create and maintain an accurate llms.txt guide with the current proposal, canonical content links, optional Markdown pages and practical validation checks.

Implement llms.txt as a concise, maintained guide to useful public content for agents that choose to read it. It can help those consumers locate documentation or business facts, but it does not replace crawl access, sitemap coverage or good page content. Publishing the file is not evidence that Google or another answer system will cite the website.

For a software agency, the useful question is practical: can a reader quickly find the services offered, delivery process, contact options and relevant technical guides without confusing old prices or archived articles with current information? An llms.txt file should make that task easier rather than becoming another enormous keyword list.

Understand the current proposal

The llms.txt proposal is now version 2, updated in August 2026. It describes a Markdown guide that can live at the root or a subpath and cover the content beneath that path. It also proposes links to clean Markdown page versions and standard link relations for discovery.

Its basic structure uses a project-name heading, an optional short summary and grouped links with useful descriptions. Keep the guide selective. A file containing every URL and thousands of repeated summaries loses the benefit of a curated starting point.

The proposal does not compel every assistant to fetch the file. Distinguish a documented consumer integration from an assumption that all major search systems follow the convention. An implementation should still be useful when a person opens it directly.

Google's guidance for AI Search features explicitly says that new AI text files are not required to appear in AI Overviews or AI Mode. It also makes clear that meeting the technical requirements does not guarantee crawling, indexing or serving.

That means missing llms.txt is not a sufficient diagnosis for disappearing Google impressions. Check the affected page's status, indexability, canonical, internal links and actual Search Console evidence first. A newly published text file cannot repair a sitewide noindex rule or a hosting firewall challenge.

Our AI search visibility audit separates technical eligibility from observed citations. Use that distinction when setting a budget: a small maintained guide can be worthwhile, but it should not displace work on inaccurate or inaccessible source pages.

Select pages by the questions they answer

Start from a handful of real tasks. A prospective client may need to know whether the agency builds a particular kind of application, how to discuss requirements and where to read a relevant engineering explanation. A developer may need API documentation or integration constraints.

For each selected link, write one sentence describing what the destination actually covers. Prefer “Scope and acceptance questions for a custom software project” over “Best number-one software development experts.” The former helps a consumer choose a source; the latter makes an unsupported promotional claim.

Use current, canonical destinations. If a comparison article has been merged into a stronger guide, link directly to the surviving page. Do not maintain a separate alternate history in llms.txt that points to removed or quarantined URLs.

An editorial selection checklist can include:

  • A clear description of the organization or product.

  • The principal service or documentation entry points.

  • Current public contact and support routes.

  • Guides answering distinct, relevant questions.

  • Limitations or eligibility facts readers commonly misunderstand.

  • Links to authoritative policies where needed.

  • A small optional section for secondary references.

  • An identified owner for keeping the guide accurate.

Keep business facts consistent across formats

Generate repeated facts from the same reviewed source when feasible. If a service price is monthly, the guide must preserve that unit. If an estimate is illustrative, do not convert it into a fixed offer by shortening the wording.

The same care applies to availability, credentials and guarantees. A text guide should not claim certifications, response times or client outcomes that the business cannot substantiate. Machine-readable copy is still public business communication.

Review content when the underlying service changes. A visible page can be corrected while a cached text file continues to publish the old statement. Include the guide in the same release checklist as navigation, structured data and pricing components.

Add Markdown alternatives only when they can stay accurate

Clean Markdown versions can help consumers that support them, especially for documentation with substantial navigation around the main content. They should preserve important caveats, source links and headings rather than extracting only the most promotional sentences.

Do not publish an automatic alternate for every page without checking its output. Tables, interactive examples, images and conditional content may lose meaning during conversion. An architecture diagram needs a useful text explanation; a pricing selector needs clear currency and billing assumptions.

If you expose alternates, maintain a reliable relationship to the canonical human-readable page and test the published responses. The proposal's discovery mechanisms are optional implementation choices, not evidence of a search ranking requirement. A small accurate guide linking to good HTML pages can be preferable to a large set of stale text exports.

Validate delivery and destinations

Fetch the deployed file directly. Confirm that it returns the expected text with a successful status and an appropriate text content type, without a login page or browser challenge. Inspect the canonical host and any subpath guide separately.

Check each destination for a successful response, relevant content and correct canonical. A redirect may work for a consumer, but a direct current URL is simpler to maintain. Treat broken links and obsolete service descriptions as release defects.

Review AI crawler access independently. The guide cannot authorize access to a page blocked by authentication or guarantee that a bot interprets your robots groups as intended. Use the technical SEO checklist for the underlying pages.

Test usefulness with a bounded exercise

Give a consumer that supports ordinary URL fetching the file as its starting point and ask a few factual questions. For example: which page explains project procurement, where can a client contact the agency, and does the pricing description specify a billing period?

Inspect the retrieved sources and answer, not only the final fluent prose. If the consumer chooses the wrong article, improve link descriptions or grouping. If the destination lacks the answer, fix the page rather than hiding a unique unsupported answer in llms.txt.

Record file fetches and referrals where available, but keep attribution modest. A fetch shows that someone requested the file. It does not establish that a subsequent citation or ranking change was caused by it.

Is llms.txt an access-control or licensing mechanism?

No. Put access restrictions in the appropriate authentication and network controls, and publish applicable terms through the organization's established process. A descriptive text file cannot enforce either boundary.

Should every article be listed?

Usually a curated guide is more useful than a complete article dump. Keep the sitemap responsible for enumerating eligible canonical pages and use topic entry points to help readers navigate a large archive.

MT

Written by

Moueen Togarvi

Founder & CEO at Voquarn Code, focused on product engineering, search growth, and practical AI systems.

About author
Turn the insight into action

Need a practical plan for your next digital project?

Tell us what you are building. We will help you clarify the scope, technical approach, and highest-value first step.

Discuss your project