If you have ever opened a pull request from a coding agent like Cursor, scrolled through a long diff and found that the only real change was one reworded sentence, you already know where this story starts.

I keep our API reference in sync with a live stage OpenAPI file. The staging environment moves fast, and the docs have to follow it before developers and their coding agents copy the old version into real client code.

If you are looking to learn more about api doc automations and agentic workflows, check out my book here: The AI Native Technical Writer

The above plan looks fine on paper, but it starts to fall apart once it runs against a real stage environment, and it fails in four specific ways.

  • Wording changes became work: Someone rewrote a description on stage. Nothing about the contract changed, but I still got a pull request to review.
  • Unrelated changes piled up: Three products changed on the same day and landed in one giant review that nobody wanted to own.
  • The same change got drafted twice: An open draft already covered Monday’s change, and Thursday’s run drafted it again.
  • A broken deploy looked like a real change: A bad deploy dropped a big chunk of routes for a while, and the model was ready to document that the API had vanished.

All four problems had the same cause. I was asking an AI to answer questions that plain code answers better, so I changed the order of the system. Now a regular Node.js watcher does the hard work. The AI only drafts after the change is already classified, and a person still merges every pull request.

This post walks through this design. It works for OpenAPI, but the same idea fits protobuf, GraphQL or any contract that moves faster than the people who keep its consumers up to date.

The real problem is contract drift

“The docs are out of date” sounds like a writing problem, but what actually goes wrong is API contract drift. The staging environment says one thing, the written contract says another and developers trust the written one, so coding agents trust it too. When the two disagree, people ship client code against a shape the API no longer has.

The usual fixes don’t help much. Waiting for someone to mention the change in Slack is too slow. Regenerating Markdown from the whole OpenAPI spec every night is too much work, because every run produces a huge diff and reviewers stop reading carefully.

Before a model writes a single sentence, I want answers to five questions:

  1. Did the contract change, or only the wording?
  2. Which product area owns this change?
  3. Is this a new endpoint, or a change to an existing one?
  4. Did we already draft this in an open pull request?
  5. Is stage healthy, or did a broken deploy just delete a lot of routes?

Each of these has a correct answer that code can compute. A model should never be guessing them.

How the pieces fit together

The system has two parts, and each one has a single job.

  • The watcher is a scheduled GitHub Action that runs Node.js. It notices changes, classifies them, splits them into groups and refuses to act when the stage looks broken.
  • The drafting agent is a Cursor Automation with strict rules. It takes one group of changes and turns it into one draft docs pull request.

Step 1: Store the API one endpoint at a time

The watcher never compares yesterday’s full JSON file with today’s as one big file. It starts by fetching the stage spec:

node watch.mjs fetch --out /tmp/stage-doc.json

Then it splits the spec into one small snapshot file per operation. An operation is one method and path pair, like these:

POST /api/v1/sessions
GET /api/v1/sessions/{id}
POST /api/v1/billing/invoices
...

Each snapshot holds the operation and the schemas it needs, so it can be read and compared on its own.

This looks like extra bookkeeping at first, but it changes how every later step behaves:

  • Small diffs. A change to one endpoint no longer creates a 5,000-line pull request.
  • A local baseline. The snapshot files travel with each pull request, so merging one updates the baseline only for the endpoints it touched.
  • Independent drafts. Open drafts for different endpoints don’t interfere with each other.

It feels much closer to database migrations than to rewriting the whole thing. Each merged pull request moves the known truth forward for a few endpoints and leaves everything else alone.

Step 2: Compare facts, not sentences

This is the part most “AI watches OpenAPI” demos skip, and in my opinion it’s the most important part of the whole system.

The watcher flattens each operation into a list of facts. A fact is a key that points to one piece of the contract:

parameter.header.X-Request-Id
request.body.amount
response.201.body.invoice_id
response.200.header.Location

Some facts describe the shape of the contract: types, required fields, enums, status codes, headers and auth requirements. Other facts are only text: summaries, descriptions and example wording.

The rule that follows is simple. If someone rewrites a description and the shape stays the same, the watcher opens nothing. If a field becomes required, it drafts a change and flags it as breaking.

Here is a simplified version of the classifier:

function classifyFact(key, before, after) {
if (!before && after) {
return {
key,
kind: "added",
isShapeChange: isShapeFact(key),
isBreaking: isNewRequiredInput(key, after),
};
}
if (before && !after) {
return {
key,
kind: "removed",
isShapeChange: isShapeFact(key),
isBreaking: isShapeFact(key),
};
}
if (JSON.stringify(before) !== JSON.stringify(after)) {
return {
key,
kind: "changed",
before,
after,
isShapeChange: isShapeFact(key),
isBreaking: typeChangedOrBecameRequired(before, after),
};
}
return null;
}

A removed shape fact always counts as breaking. That’s deliberately cautious, because some client may still send or read that field. A deleted description, on the other hand, is only text and can’t break anyone.

Two stop rules run before the watcher pushes any branch:

if (report.removedRatio > 0.25) {
throw new Error("Stage looks broken. No PRs.");
}
if (onlyTextFactsChanged(report)) {
return { status: "description-only" }; // do nothing
}
The second rule handles description-only runs, which are a perfectly good outcome. The first rule is the one I’m most glad I wrote.

A bad deploy can make half the routes disappear for a while. A model will cheerfully document that as a real removal. A ratio check in plain code refuses, fails the run and opens nothing.

The watcher also looks for renames. When POST /old disappears, and POST /new appears with enough of the same shape, it reports one rename instead of a deleted endpoint plus a new one. Reviewers care about that difference, because a rename and a removal need very different follow-up work.

Step 3: Split the work the way people review it

When a real change does show up, the watcher doesn’t open one mega pull request. It groups the changes by two things:

  • Product area, such as auth, billing or search.
  • Kind of change, which is either new endpoints or changes to existing ones.

Each group gets its own branch:

auto/stage-api/auth-new
auto/stage-api/auth-existing
auto/stage-api/billing-new
...

I chose this split because of how review actually happens on a team:

  • Someone can merge auth without waiting on billing.
  • New endpoints need pages and navigation entries, while existing endpoints need field updates and callouts. Mixing the two makes a review painful to follow.
  • Two agent runs can work in parallel without fighting over one branch.

The routing config stays boring on purpose:

{
"stageSpecUrl": "https://stage.example.com/api/docs/v1/doc.json",
"maxRemovedRatio": 0.25,
"products": [
{
"id": "billing",
"label": "Billing APIs",
"match": ["/api/v1/billing", "/api/v1/invoices"],
"folders": ["billing-api-reference"],
"changelog": "billing-api-reference/changelog.mdx"
}
]
}

The watcher matches each path against these prefixes, and the longest matching prefix wins. Anything that matches nothing falls into a default area, so an unmatched change still gets drafted somewhere. You can use the same idea in any repo that documents several services.

Step 4: Treat open drafts as part of the truth

This is usually where a weekend demo stops working.

Imagine that Monday’s run opens this pull request:

[Stage API] Billing APIs: Changes to existing endpoints (2026-11-09)

Nobody has merged it by Thursday, and stage changes the same endpoint again. If Thursday’s run only compares stage with main, it sees Monday's change as new and drafts it a second time.

So before the watcher diffs anything, it layers every open [Stage API] pull request on top of main:

baseline = main
+ open PR #101
+ open PR #108

then compare stage against that baseline

That gives two possible outcomes:

  • If the open pull request already covers the change, Thursday’s run does nothing.
  • If stage moved again after that pull request, Thursday’s run drafts only the newer delta. Its commit message says Also changed in open PR: #101, and the agent puts "Builds on #101. Merge #101 first." at the top of the new pull request.

None of this needs a clever model. The system only has to remember which work is already in flight.

Step 5: Claim the group, then read the brief

A push to a group branch starts a Cursor Automation run. The agent doesn’t get a vague instruction like “go fix the docs”. It gets a short protocol instead.

Claim the group

First, the agent claims the group so that two runs never draft the same work:

git checkout --detach origin/auto/stage-api/billing-existing
git commit --allow-empty -m "stage-api-claim: billing-existing"
git push origin HEAD:refs/heads/auto/stage-api/billing-existing

If the push is rejected, another run already owns the group, so this run stops.

This works because both runs start from the same branch tip. The first claim to land moves the branch forward. The second claim is now built on an old tip, so git rejects it as a non-fast-forward push.

One more rule closes the gap. The claim itself is a push to the group branch, so a run that starts after it will find a claim commit at the tip. The protocol tells that run to stop as well.

It’s a tiny lock made of git. It’s simple; anyone can see it in the branch history, and it’s enough for this job.

Read the brief

Next, the agent asks the watcher for a brief:

node watch.mjs brief \
--base origin/main \
--label billing-existing \
--md /tmp/brief.md \
--json /tmp/brief.json

For each endpoint in the group, the brief tells the agent:

  • change: whether the endpoint is new, changed, renamed or removed.
  • existingPages: the pages that already document this operation.
  • placement: where a new page should go.
  • inOpenapiJson: whether the shared OpenAPI file already has it.
  • changes: the before and after facts, with breaking flags.

Then the rules get strict.

  • The agent may edit the shared OpenAPI file, the matching endpoint pages, the navigation for new pages and that area’s changelog.
  • It must not wander into concept guides, tutorials, SDK folders or site chrome.
  • For removed endpoints, it never deletes pages. It flags them for a person instead.
  • It always opens a draft pull request.

That last rule is the point of the whole design. The model drafts, and it never ships.

What a useful draft PR looks like

The title is part of the protocol:

[Stage API] Billing APIs: New endpoints (2026-11-13)

The [Stage API] prefix is how later runs recognize open work, which is exactly what Step 4 depends on.

The body is written for the reviewer, not for show:

> Drafted from the stage OpenAPI file.
> Stage may be ahead of production. Merge when the change is real.

**Area:** Billing APIs · **Kind:** New endpoints

## What changed on stage
- `POST /api/v1/billing/invoices`: new
- Breaking: none

## What I changed
- `openapi.json`: added operation and schemas
- `billing-api-reference/invoices/create.mdx`: new page
- changelog: vX.Y.Z

## Needs your input
- Confirm the changelog version if another Stage API PR merged first
- Placement was unclear for one path

## Other pages that mention these APIs
- `guides/quickstart.mdx:42` (not edited)

## Checks
| Check | Result |
| :--- | :--- |
| Validate docs build | Pass |
| Style lint | Pass |

One more rule saved me here. The agent doesn’t quietly rewrite every guide that mentions the endpoint. It lists those files under “Other pages that mention these APIs”, and a person decides what else should change.

Why AI can automate this isn’t enough

Anyone can wire this up in an afternoon now:

schedule → fetch OpenAPI → prompt a model → open a PR

That pipeline is easy to build. It’s also a very fast way to generate review noise.

What made my version useful was everything around the model:

  • Shape changes over wording changes, so description edits stop being fake work.
  • One file per endpoint, so merges stay small and local.
  • Groups by product area and change kind, so a reviewer can approve one thing without owning five.
  • Open PRs counted as known work, so the same change never gets drafted twice.
  • A refusal to trust a broken stage, so a bad deploy never gets documented.
  • A claim before drafting, so two agents never fight over one group.
  • A short brief and a short edit list, so the model works inside clear limits.
  • Draft PRs only, so stage can lead while the docs still wait for a person.

This pattern also works outside docs

You can use the watcher, swap what it drafts and apply the same pattern for other contracts.

Whatever the contract is, the steps stay the same:

  1. Snapshot the contract in small pieces.
  2. Compare shape, not prose.
  3. Group by owner and change kind.
  4. Layer open work before comparing.
  5. Give an agent a brief and clear limits.
  6. Open a draft for a human.

Here is a small starter kit you can build from:

const stage = await fetchSpec(STAGE_URL);
const base = await readSnapshots("snapshots/");
const open = await loadOpenAutomationPrs();
const layered = applyOverlays(base, open);
const report = compareFacts(layered, stage);

if (report.removedRatio > 0.25) fail("broken deploy");
if (!report.hasShapeChange) exit("nothing to do");

for (const group of groupByOwnerAndKind(report)) {
writeSnapshots(group);
pushBranch(`auto/contract/${group.id}`);
// agent reads brief(group) and opens a draft PR
}

You don’t even need the agent on day one. A Markdown brief inside a pull request is already better than hoping someone notices that stage changed.

What I learned once it worked

  • Most runs should do nothing: Unchanged and description-only runs are good outcomes. A quiet watcher means the contract and the docs agree.
  • Agents copy nearby pages well: When I give the agent the closest sibling page and tell it to match that structure, the result is much better than asking it to invent a page style.
  • Deleted endpoints are a people problem: The watcher detects the removal, but it never deletes the page. Someone still has to add redirects and confirm that the endpoint is really gone in production.
  • Changelogs collide: Two groups can bump the same changelog on the same day, so a person confirms the version number at merge time.
  • The title prefix matters: If someone renames a pull request and drops [Stage API], the watcher can't see that work and may draft the same change again. The naming rule is part of the system, not decoration.

Conclusion

AI made documentation updates easy to generate, but it didn’t make them easy to trust, and trust is what decides whether anyone merges them.

That’s why I think of a stage API watcher as a reliability tool more than a docs tool. So if you’re building something in this space, don’t start with the prompt; instead, count the real change, avoid duplicate drafts, and decide what the agent is not allowed to touch. The AI opening a pull request was never the interesting part, but the interesting part is that the pull request was already constrained before the model generated its first token.

How do you keep API docs, SDKs or clients up to date with a contract that changes faster than your team? I’d love to hear what has worked for you, so tell me in the comments.


Building Self-Maintaining API Documentation with AI Agents was originally published in Level Up Coding on Medium, where people are continuing the conversation by highlighting and responding to this story.