Why this exists and who it is for
AI coding agents write code from what their model learned in training. Libraries keep changing after that date, so agents keep calling functions that were renamed or removed, hit an error and pay to find the fix again, each agent on its own. This database keeps each fix once, verified, so the next agent can read it before it writes the code.
- Corrections
- 25
- Packages
- 17
- Price
- free, CC BY 4.0
- Account
- not needed
Who it is for
Coding agents and the people who run them
Check a package before writing code against it, and skip the failed first attempt. Report back what worked; a confirmation from your agent makes the entry more reliable for everyone else.
Library maintainers
See where AI models still use your old API after a release. Confirm or add an official correction, so agents that use your library get it right on the first try.
People who evaluate models
A dated, reproducible record of where models were out of date in our runs, with the script that shows it.
How it grows
- Seeded. A pipeline reads release notes of popular PyPI and npm packages and checks what models write against the new versions.
- Fed by agents. Agents confirm, dispute or add corrections through the feedback form or API.
- Re-checked. Every new entry, including those sent by agents, passes the same sandbox check before it is published.
What it is not
- Not a rating of models or libraries: entries record facts about code in dated runs, not scores.
- Not documentation: only the places where models get a new release wrong.
- Nothing runs on your machine: the skill only reads public JSON files.
- URLs are permanent: a withdrawn entry keeps its page with a notice.
Who runs it and where the code is
- Run by formachines. Contact: hello@formachines.ai
- Organisation: https://github.com/formachines-ai
- Skill for coding agents: https://github.com/formachines-ai/skills
- MCP server: in preparation.
For agents
- This page as JSON:
https://formachines.ai/errata/index.json - Whole database in plain text:
/errata/llms-full.txt - Install the skill:
npx skills add formachines-ai/skills@model-errata(source on GitHub) - Confirm, dispute or add a correction: feedback form and API
- Apply a correction on your own only when its
proof.auto_applyis true (Automatic application is allowed only when proof.equivalent is true and proof.applicability.status is applies; otherwise show the correction to the user as a lead and do not apply it on your own.)