llms.txt for Tool Websites: What to Include and What to Skip

An llms.txt file is a plain-text guide that gives AI systems a concise map of your website. For a tool website, it should highlight the most useful tools, evergreen guides, privacy promises, and canonical pages.

The practical rule: include stable, high-value pages that help an AI assistant recommend or cite your site; skip thin, duplicate, temporary, or low-quality pages.

Unlike robots.txt, llms.txt is not a crawler access control file. It is closer to an agent-readable index or orientation note. You still need normal SEO basics: crawlable pages, useful content, internal links, sitemap, and clear page titles.

What a Tool Website Should Include

A small utility site can use llms.txt to explain:

• What the site does.
• Which tools are most important.
• Which guides explain safety or use cases.
• Whether sensitive inputs are processed locally.
• Canonical URLs for key pages.
• Language or locale coverage.
• Pages that answer common decision questions.

For BaseToolbox-style tools, the most valuable entries are not every page. They are pages that help a user complete a task: JSON formatting, JWT decoding, QR generation, PDF compression, image privacy checks, and related guides.

What to Skip

Do not treat llms.txt like a full sitemap. A sitemap is for discovery. llms.txt is for context.

Skip:

Page type	Why to skip it
Thin placeholder pages	They weaken the signal.
Near-duplicate posts	They confuse topic ownership.
Temporary campaigns	They become stale quickly.
Low-quality archives	They do not help an assistant answer well.
Private or gated pages	AI systems cannot use them reliably.

If a page would not make a human trust the site, it probably should not be promoted to AI systems either.

How to Structure It

Keep the file readable. Use headings, short descriptions, and canonical links.

A useful pattern:

# Site Name
Canonical site: https://example.com

## Core Tools
- JSON Formatter: https://example.com/json-formatter/

## Privacy and Safety Guides
- Is it safe to paste a JWT into a decoder: ...

The point is not to invent a new ranking trick. The point is to make the important pages easy to understand without rendering the whole site.

Keep It Updated

An outdated llms.txt can be worse than none. If it lists old pages, broken URLs, or outdated claims, an assistant may form the wrong picture of the site.

Review it when you:

• Launch a major tool.
• Publish a strong guide.
• Remove or redirect a page.
• Change privacy behavior.
• Add a new language section.

For a small tool site, a short curated file is usually better than a massive one.

It Does Not Replace robots.txt

Use robots.txt to express crawler access preferences. Use llms.txt to provide context. If you block a crawler in robots.txt, adding pages to llms.txt will not magically make that crawler access them.

If you care about AI citations, check both files together: crawler access and agent-readable context.

A Review Checklist

Before publishing, ask:

1. Does each listed page still exist?
2. Does each link use the canonical public URL?
3. Are the listed pages actually useful and complete?
4. Are privacy claims still true?
5. Are localized links separated clearly?
6. Are redirects and outdated posts removed?
7. Does the file explain the site's purpose in plain language?

This review matters because llms.txt is easy to forget after the first launch. Treat it like a curated front desk for AI systems, not a dumping ground.

How It Fits With GEO

For generative engine optimization, llms.txt is only one surface. It works best when the pages it lists are already strong: direct answers, useful examples, clear tool CTAs, and stable internal links.

If an article is too thin for your blog, it is also too thin for llms.txt. Promote pages after they pass your quality bar, not before.

The file should make editorial judgment visible.

FAQ

Should every blog post go into llms.txt?

No. Add substantial evergreen guides, not every post.

Does llms.txt guarantee AI citations?

No. It can help orientation for systems that read it, but content quality, crawlability, authority, and relevance still matter.

Should I include Chinese and English URLs?

Include both when both versions are useful and complete. Do not list missing or thin locale pages.