Readmes for LLMs
Most people drastically underinvest in understanding how LLMs differ from humans. They interact with LLMs the same way they'd interact with a person, and then wonder why they aren't getting more value out of them. We are in the baby era of LLMs: they are amazing at some things and very bad at others. Communication style matters a lot.
Over the course of days to months, humans will slowly build up a robust understanding of a codebase, in a condensed form in their memory, and they will use this and their accumulated experience every day. If a document is too long, they may not read it, but it is unlikely to distract them from other documents. LLM's context windows can get bogged down by overly long documents, and they can't slowly build up intuition over months: they start fresh every task.
Humans get bored of reading fast, and they need to be entertained to engage them. A piece of writing that is a bit longer but is more engaging will be better at information transfer with humans. LLMs don't have the boredom problem, and thus ruthless conciseness is almost always better.
Every time a human has to click a link or find another page, there is a strong chance of losing them. But an LLM will follow the link if it seems worthwhile to their task pretty reliably.
Humans often benefit from explanation of how a given block of code works even when it's right in front of them (however, I think this is often overdone in commenting norms). LLMs do not benefit from this at all, they understand code as well as they understand language, and code is more precise.
Given this, it's clear that there is a preferable way to write a top level readme in this new era of LLMs.
LLMs have increased the value of conciseness in a top level readme. Conciseness has always been valuable, and I think most readmes are too long, but in the past a multi-section decently long readme could be good, especially if it was interesting to read. Nowadays, I think it's best that the top level readme is as short as possible and mostly a collection of signposts to other readmes about specific things. That way, the top-level readme can be fed to an LLM at the start of every session, without including unnecessary specifics.
I don't think that it makes sense to try and maintain two sets of documentation: one human-optimized and one LLM-optimized, unless the number of human users is high enough (>20 or so). And the only difference for a human optimized version is that additional sections could be added under this, I think this is the correct start to any readme.
Here is how you should write a top-level readme for an LLM.
Top-level Readme Template
Preamble
High level description of the repo. What it is for, major design considerations and goals.
Environment
Information about the environment the code is meant to run in, also pointer to installation scripts. I would avoid putting specific install instructions, and instead put that in an installation readme.
Style Notes
Optional, but if there are repo-specific styles to follow. If in doubt, remove this section
Entry point table
Entry point table listing all the used entry points into the codebase with minimal description and a pointer to more info. All paths in the table should be relative to the repo root.
| Entry point | Description | Additional readme |
|---|---|---|
bash dev.sh | Start local dev server | - |
python -m pytest | Run test suite | tests/README.md |
bash deploy.sh | Deploy to production | docs/deploy.md |
python app/main.py --help | CLI tool usage | docs/cli.md |