Skip to content
Starlight Pydocs

llms.txt

Each documented package serves its whole API surface as plain Markdown at <base>/llms.txt. On this site, that is /starlight-pydocs/api/demopkg/llms.txt. The endpoint is on by default. Set llmsTxt: false to remove it from every package.

A single Markdown file suits a language model better than a crawl of rendered HTML. It also makes a good diff target: the package’s own unit tests snapshot this same format, so a change in the rendered surface shows up as a change here too.

What it contains

Section titled “What it contains”

The file starts with a short header that names the package and links to the rendered pages. After that comes every page of the package, in navigation order. The example below is abridged, from demopkg:

# demopkg


> API reference for the Python package `demopkg`, generated by starlight-pydocs.
> Rendered pages: https://ewels.github.io/starlight-pydocs/api/demopkg/


# demopkg


*module*


Demo package used by the starlight-pydocs test suite.


## demopkg.Report


*class*


```python
class Report(BaseReport)
```


Bases: `demopkg.report.BaseReport`


Re-exported from: `demopkg.report`


A named collection of scored sections.


**Parameters**


- `name` (`str`) — Human readable report name, also used as the file stem.
- `scores` (`dict[str, float] | None`) (default: `None`) — Mapping of metric name to score. Defaults to an empty mapping.


**Attributes**


- `name` — The report name.
- `scores` (`dict[str, float]`) — The scores passed to the constructor.


[View source](https://github.com/ewels/starlight-pydocs/blob/main/fixtures/demopkg/src/demopkg/report.py#L57-L160)

Every object follows the same layout, shown as four highlighted runs above. The heading is the dotted path. Italics give the kind and any labels. A python fence holds the signature. Next comes provenance: bases, re-exports, alias targets and deprecation notices. After that come the docstring sections. The source link comes last, after everything the docstring produced and before the object’s own members.

Heading depth follows the rendered page, so a method sits one level below its class. Doctest blocks are fenced as pycon. Docstring prose passes through as written, so cross-references keep their [title][dotted.path] form. A model can use that form better than a resolved URL.

The file is prerendered, so a static host serves it as text/plain regardless of the text/markdown the route sets. Browsers show it, curl pipes it, and models read it.

One page at a time

Section titled “One page at a time”

The same Markdown is served per page, at the page’s own path plus .md. The page at /api/demopkg/report/ is also /starlight-pydocs/api/demopkg/report.md, and a package’s root page answers at its base: /starlight-pydocs/api/demopkg.md.

Each one is served again at .md.txt, for example /starlight-pydocs/api/demopkg/report.md.txt. The bytes are identical. Only the extension differs, and it is the extension a host reads: .md.txt gets text/plain, which browsers and viewers show inline, where a .md is often downloaded instead. Link the .md for machines and the .md.txt for people.

Use the whole-package file to give a model an entire API in one request. Use the per-page file when something already holds a single page’s URL: an agent handed a link, or a reader who wants this page and not the other twelve.

Each generated page advertises its own .md in the document head, so nothing has to know the convention to find it:

<link rel="alternate" type="text/markdown" href="/api/demopkg/report.md" />

Set pageMarkdown: false to remove all of these routes, and that tag with them.

<path>.md is also the convention the Starlight page-action plugins fetch, so their buttons work on the generated pages as well as on the ones you wrote. This site runs starlight-page-actions, which is where the Copy Markdown and Open buttons under every page title come from:

astro.config.ts
starlight({
  plugins: [
    starlightPydocs({ packages: [{ name: 'demopkg', search: ['../fixtures/demopkg/src'] }] }),
    // Copies `src/content/docs/**` to `<path>.md` and adds the buttons. The
    // generated pages are not files there, but they answer at `<path>.md`
    // themselves, so the same buttons work on both.
    starlightPageActions({ position: 'page-title' }),
  ],
});

Wiring it into starlight-llms-txt

Section titled “Wiring it into starlight-llms-txt”

starlight-llms-txt builds a site-wide llms.txt from the docs content collection, the Markdown files under src/content/docs/. The generated API pages are not files there. This plugin injects them as routes when Astro builds. Nothing lists those routes for starlight-llms-txt, so it cannot see any of them. That is why you must tell the two plugins about each other, using the optionalLinks option: it takes a list of URLs directly, rather than reading pages to find them.

This site runs both, so you can click through the result:

The configuration behind that:

astro.config.ts
import starlight from '@astrojs/starlight';
import { defineConfig } from 'astro/config';
import starlightLlmsTxt from 'starlight-llms-txt';
import starlightPydocs, { pydocsSidebarGroup } from 'starlight-pydocs';


const SITE = 'https://ewels.github.io/starlight-pydocs';


export default defineConfig({
  site: 'https://ewels.github.io',
  base: '/starlight-pydocs',
  integrations: [
    starlight({
      title: 'Starlight Pydocs',
      plugins: [
        starlightPydocs({
          packages: [{ name: 'demopkg', search: ['../fixtures/demopkg/src'] }],
        }),
        starlightLlmsTxt({
          optionalLinks: [
            {
              label: 'demopkg API reference (Markdown)',
              url: `${SITE}/api/demopkg/llms.txt`,
              description:
                'The example package, rendered by this site: google-style docstrings, pydantic models, deprecations, inheritance and `__all__` filtering.',
            },
          ],
        }),
      ],
      sidebar: [{ label: 'API reference', items: [pydocsSidebarGroup] }],
    }),
  ],
});

Each entry takes label, url and an optional description. It lands under the ## Optional heading of the generated llms.txt, where a model can follow it when it wants more than the prose. Build the URLs from the site origin, as SITE does above, because clients that read this file do not know your site’s base path.

Add one entry per package you want a model to find, and no more. This site documents demopkg twice: once extracted from the working tree, and once from a pre-generated dump at 1x/api/demopkg that stands in for an older release. It advertises only the current one, so a model never lands on an old release when it answers a question.

Consuming it yourself

Section titled “Consuming it yourself”

The endpoint is a normal prerendered file, so anything can read it:

curl -s https://ewels.github.io/starlight-pydocs/api/demopkg/llms.txt > demopkg-api.md

If you want the same information as structured data instead of Markdown, use the Content Layer loader. It exposes one entry per documented object, with its path, kind, brief, signature and page. Use it to build a table, a cheat sheet, or your own index.

Listing the generated pages

Section titled “Listing the generated pages”

Site-wide features that iterate over pages — share card images, a custom index, an extra sitemap — build their list from your content collection. The generated pages are injected routes, so they are not in it. Ask the plugin for them instead:

src/pages/og/[...route].ts
import { getCollection } from 'astro:content';
import { listPydocsPages } from 'starlight-pydocs/pages';
import context from 'virtual:starlight-pydocs/context';


const generated = await listPydocsPages(context);
// [{ base: 'api/demopkg', slug: 'api/demopkg/report',
//    title: 'demopkg.report', description: 'Report classes and the …' }, …]


const pages = {
  ...Object.fromEntries((await getCollection('docs')).map((entry) => [entry.id, entry.data])),
  ...Object.fromEntries(generated.map((page) => [page.slug, page])),
};

The slug is the page’s path within your site, with the package base already on the front, so it works as a route key and as a URL. The description is the module’s first docstring line, which is also what the page itself uses for its <meta name="description"> and OpenGraph description.

This site generates its own share cards this way. The astro-og-canvas route reads both lists, and a Head component override points each page at its image.