The finding · 25 August 2026
Most AI agent failures are a property of the repository, not the model.
We marked the 864 most-starred software repositories on GitHub against a fixed, published set of checks. Nothing was cloned and no code was run. Here is what came back.
Repositories that ship an AGENTS.md score better on everything else too
Read that carefully, because it is the most misreadable number here. Writing the file does not earn a repository 24 points. Teams that stop to write down how their project works are the same teams that already committed a lockfile and made the tests runnable. The file is a symptom of that habit as much as a cause. It is still the cheapest place to start.
Almost nothing is ready
Eighteen repositories out of 864 scored an A. Just over a third landed below 55, which is the point where an agent spends more turns orienting itself than changing anything.
Marks across 864 repositories
grade, count, share of the total
Higher is better. Curated lists and tutorials excluded.
The weak link is always the same one
Repositories are documented, licensed, and reasonably organised. What they do not do is tell an agent how to work in them. Instructions is the worst category by a distance, and it is the cheapest one to fix.
Mean score by category
percentage of available marks earned
Instructions is worth 27 of the 100 marks.
Three numbers worth sitting with
Language matters more than it should
A TypeScript repository is not better engineered than a C one. What it has is a package manager that produces a lockfile, a convention for where tests go, and a scripts block that names commands. Ecosystem defaults do most of this work, which is why the spread is this wide.
Median mark by language
languages with at least 15 repositories
Sample sizes: TypeScript 173, Go 77, Rust 57, Python 214, JavaScript 104, Shell 15, Java 40, C++ 41, C 21.
Popularity predicts almost nothing
The hundred most-starred repositories have a median of 66 against 61 for the rest. Five points. Being loved is not the same as being workable, and the projects an agent will struggle with most are often the ones it is most likely to be pointed at.
How this was measured
Two API calls per repository. Metadata and the full file tree. Small configuration files come from a CDN. No repository was cloned and no code was executed.
Lists and tutorials are excluded. Many of the most-starred repositories on GitHub are curated link collections and courses. Asking whether an agent can install a list of links means nothing, so 136 of the 1,000 were marked separately and left out of every number here.
Checks that do not apply are not counted. A library has no environment to configure. Those are removed from the total rather than scored zero, so nothing is punished for being what it is.
The scheme is opinionated and some marks are wrong. Detection misses things, particularly outside JavaScript and Python. Every rule is open source, every deduction cites the file it came from, and a pull request beats an argument.
See where yours lands.
The median is 61. It takes about two seconds to find out.