Process re-engineering with AI: when review is the bottleneck

The problem: writing got cheap, checking did not

Most build processes were designed around the slow part. When writing code took days, a review could be a skim, because the reviewer had time to read carefully later. An agent can produce a week of code in an afternoon. The process still assumes writing is the constraint. It isn’t. Reading and deciding are.

The old process, and where it breaks

The default flow is: task, agent writes, you skim the agent’s summary, you merge. It is fast. On one busy day, an agent-heavy run of commits on my Datastar library built most of the Dstar.Page layer. Speed was never the problem. The problem is that a summary describes the work, and only the diff shows what landed.

The re-engineered process

Four changes, each aimed at a specific failure of the default flow.

1. Define done before the work starts. Each task gets a checkable condition: which test runs, what must be observable, what must not change. Vague instructions give you confident, well-formatted, wrong output.
2. Read the diff, not the summary. The diff is what actually landed. The summary is the agent describing its own work.
3. Label every finding and record its verdict. Each finding is fixed, retracted, or deferred, with a reason. The dstar commit history carries the labels: “Fix SSE frame injection via carriage returns (S1)” and “Fix a stale takeover signal poisoning the next stream (review F1)”. The label is how you can see that a review happened.
4. Allow retraction. If a finding is wrong, remove it. The commit “Retract the connect() placement warning — it was not a real bug” is the one I’m proudest of, because the agent had shipped a false alarm and removing it was real work.

The agent writes most of the code. The human owns the judgment and the record of it.

What the review caught

The dstar history is the evidence I have. Most commits in that library were agent-involved, and a large share landed in a few busy days. Those numbers measure output. They say nothing about checking, so the useful part is what the review caught:

• An SSE frame injection through carriage returns, which could break frame boundaries. It was one line in a large diff and did not announce itself.
• A false-alarm warning about connect() placement, retracted before anyone chased it.
• A stale takeover signal that poisoned the next stream.
• A CSRF claim in the docs that was wrong, and got corrected.

My read is that a test would not have caught the streaming issue on its own. It surfaced because the review was a step with a checklist, not a skim. That is one case, not a measured rate.

What I am not claiming

I have not timed the default process against the re-engineered one, so this post doesn’t claim hours saved. The metric I can defend is how many findings got a verdict before merge.

Where this breaks

• A tired reviewer drifts back into skimming. The checklist helps, but it doesn’t replace attention.
• A process change with no verification condition moves the mess downstream. You find the bugs later instead of now.
• Labels only matter if someone reads them. A list of verdicts nobody revisits is paperwork.

A simple exercise

Pick one process you run with an agent this week and write down where the work now piles up. If it’s review, write one done-condition per task before the next run, and give every finding a verdict. Run that for two weeks before changing anything else.