How to Create an llms.txt File: Step-by-Step Tutorial (2026)

Implementing an llms.txt file is technically straightforward, but requires careful execution to satisfy standard validators and crawler prerequisites. This practical guide covers everything from inspecting competitor files and configuring server headers to copy-paste code snippets for modern web frameworks.

How to See the llms.txt File of Any Website

Before deploying a custom implementation, it is highly beneficial to audit how industry leaders structure their files. To see the file of any website, users simply append /llms.txt to the root domain (e.g., https://www.anthropic.com/llms.txt or https://www.cloudflare.com/llms.txt).

Because some browser configurations attempt to render plain text files improperly or download them as binaries, developers should inspect files utilizing command-line utilities:

# Check HTTP status code, Content-Type, and response headers
curl -I https://example.com/llms.txt

# Print raw, unformatted Markdown directly to your terminal
curl -s https://example.com/llms.txt

llms.txt File Encoding, MIME Types & HTTP Headers

As established in the technical specification, incorrect header configuration will render the file invisible to AI engines. Regardless of the deployment platform, the server must be configured to output specific HTTP parameters:

• Strict UTF-8 Encoding: Must be encoded in UTF-8 without corrupt byte sequences to ensure seamless LLM tokenization.
• HTTP 200 OK Status: Redirects (301/302) or authentication blocks (401/403) will halt agent crawlers.
• Correct MIME Type: The payload must be served as Content-Type: text/plain; charset=utf-8 or Content-Type: text/markdown; charset=utf-8.
• HTTP Link Header: Announce the file globally on all web pages using: Link: <https://www.llms-text.com/llms.txt>; rel="describedby".

Copy-Paste Code Snippets for Modern Frameworks

For dynamic web applications, hardcoding a static text file is less efficient than generating it programmatically. The following implementations generate compliant llms.txt files across various stacks:

1. Next.js (App Router)

In Next.js, create a Route Handler at app/llms.txt/route.ts. Using force-static combined with Incremental Static Regeneration (ISR) ensures high performance and edge caching:

// app/llms.txt/route.ts
import { NextResponse } from 'next/server';

export const dynamic = 'force-static';
export const revalidate = 3600; // Hourly ISR

export async function GET() {
  const content = `# Project Documentation
> High-performance platform built for AI agent integration.

## Core Features
- [Getting Started](https://example.com/docs/start.md): Initial setup and configuration.
- [API Reference](https://example.com/docs/api.md): Complete RESTful API endpoints.

## Optional
- [Changelog](https://example.com/changelog.md): Historic version releases.
`;

  return new NextResponse(content, {
    headers: {
      'Content-Type': 'text/plain; charset=utf-8',
    },
  });
}

2. Astro

Astro handles root-level text files effortlessly using static file routing. Create src/pages/llms.txt.ts:

// src/pages/llms.txt.ts
export async function GET() {
  const markdownText = `# Astro Site Overview
> A blazing fast static site builder.

## Guides
- [Routing](https://example.com/routing.md): How routing works.`;

  return new Response(markdownText, {
    headers: { 'Content-Type': 'text/plain; charset=utf-8' },
  });
}

3. Static HTML & Webflow

For traditional static hosting or Webflow, open a text editor, create a new file, and save it strictly as llms.txt in lowercase with UTF-8 encoding. Upload this file directly to your website's root public_html or public directory via FTP or your hosting control panel.

4. WordPress

In WordPress, you can inject the file programmatically via functions.php using URL rewrite rules:

// Add to functions.php
add_action('init', function() {
    add_rewrite_rule('^llms\\.txt$', 'index.php?llms_txt=1', 'top');
});

add_filter('query_vars', function($vars) {
    $vars[] = 'llms_txt';
    return $vars;
});

add_action('template_redirect', function() {
    if (get_query_var('llms_txt')) {
        header('Content-Type: text/plain; charset=utf-8');
        echo "# Brand Site\n> Corporate blog and services.\n\n## Services\n- [Consulting](https://example.com/consulting): Strategy docs.";
        exit;
    }
});

5. Cloudflare Workers

For enterprise edge deployments, a Cloudflare Worker can intercept the route and serve the file globally with ultra-low latency:

export default {
  async fetch(request, env, ctx) {
    const url = new URL(request.url);
    if (url.pathname === '/llms.txt') {
      const content = `# Edge Platform\n> Global routing network.\n\n## Docs\n- [Setup](https://example.com/setup.md): Deployment steps.`;
      return new Response(content, {
        headers: { 'Content-Type': 'text/plain; charset=utf-8' }
      });
    }
    return fetch(request);
  }
};

Step-by-Step Implementation Methodology

1. Step 1: Content Curation and Auditing: Review your sitemap and curate the list down to 10 to 20 highest-value, evergreen URLs (product features, documentation, architecture, pricing).
2. Step 2: Structural Drafting: Write your single H1 tag, craft your blockquote summary, group links under 4 to 7 logical H2 categories, and ensure every link includes a 10-to-20 word description.
3. Step 3: Generate Clean Markdown Twins: Wherever possible, provide pure .md versions of your pages without HTML wrappers.
4. Step 4: Deploy and Configure Headers: Upload the file to your root domain and inject the HTTP Link header on your primary HTML templates.
5. Step 5: Validation and Testing: Verify the live file using the llms.txt Validator to ensure perfect standard compliance.

Create Your llms.txt File in Minutes

Don't want to build it manually? Use the free llms.txt Generator to automatically crawl your domain and export standard-compliant llms.txt and llms-full.txt files instantly.