// post

When agents write code faster than humans can review it

// Agentic coding moved the bottleneck from implementation to review and accountability. What a redesigned review pipeline needs, what vincent already does about it, and the parts I have not solved.

Most of the conversation about coding agents is about how much faster implementation gets. A task that used to take several days lands in hours. Large refactors stop being intimidating. Tests arrive alongside the implementation, and an agent can work through a well-defined backlog at a pace that would have looked unrealistic a few years ago.

The code arrives faster. Someone still has to decide whether it can be trusted, and that decision is becoming the bottleneck. It is also the problem I spend most of my time on now, in vincent, the agent workflow orchestrator I build.

The bottleneck moves

Software delivery has usually been constrained by implementation capacity. A team has a backlog, engineers pick up tasks, they design and implement and test and open pull requests, they review each other’s work, and it merges. How much code engineers could reasonably write put a natural ceiling on how many pull requests entered the system at all.

Agentic coding lifts that ceiling and leaves everything downstream where it was. One engineer working well with agents produces several meaningful changes in the time one used to take, and in some setups agents work through tasks semi-autonomously. Five engineers can reach the implementation throughput of a much larger team. The number of people capable of thoughtful review is still five.

So the pipeline stops looking like idea, development, review, merge, and starts looking like idea, very fast development, a very large review queue, and a merge whenever somebody gets to it. The bottleneck did not disappear. It moved.

Two stacked pipeline diagrams. In the first, idea, development, review and merge are joined by a flow of even width. In the second, the flow out of development is four times wider, a pile of unreviewed pull requests sits in front of review, and the review stage is unchanged
The boxes did not change. Only the width of the flow into them did, and review is the one stage that cannot be widened by buying more agents.

The accountability question

I worked on a product in the cybersecurity domain with a substantial engineering backlog, used in environments where reliability and security mattered enormously, including critical infrastructure. Like a lot of teams adopting AI-assisted development, there was pressure to increase delivery speed dramatically, and the reasoning from the management side was fair. If engineers generate code much faster with agents, why shouldn’t finished features move through the organization equally fast?

The ask was not only for more finished features either. Two of the headline engineering KPIs were number of commits and number of merged pull requests. Those are among the worst things you can put on a performance dashboard even in a world with no agents in it. They count output instead of outcome, they are trivially inflatable by splitting one change into six commits or one feature into four pull requests, and they charge the cost of that inflation to somebody else, because every extra pull request is another context switch for a reviewer whose time the metric does not track. Nobody sets out to game them. People just notice which shape of work makes the number go up.

Coding agents take that from a bad metric to a meaningless one. Commits and merged pull requests are now close to free, and measuring an organization on a quantity an agent can produce at near zero cost is measuring the agent and calling it engineering performance. Worse, the pressure it generates lands entirely on the part of the pipeline that did not get cheaper. If the number that gets you promoted is merged pull requests, then a human review step is the thing standing between you and your KPI, which is a genuinely dangerous incentive to hand a team shipping security-sensitive software. Delivery metrics that survive contact with agents have to describe what reached production and held up there: lead time, change failure rate, how long a defect takes to get back out. Agents make those harder to fake, not easier.

That is where the push ran into the engineers, because implementation was only ever one part of the responsibility. There were serious discussions about letting agents review and effectively approve agent-generated changes. On pure throughput that is attractive: one agent writes the code, another reviews it, and the human review bottleneck appears to vanish.

The engineers were considerably less enthusiastic, and what they kept coming back to was not a technical objection.

Who takes responsibility for code that no human has ever reviewed?

Letting an agent produce and approve a UI adjustment or an internal productivity tool is one thing. Shipping security-sensitive code into systems that organizations running critical infrastructure depend on, while stating that no engineer ever inspected the implementation, is another. The people pushing back were not resisting AI. They were resisting the idea that faster code generation moves engineering accountability somewhere else. That distinction is the entire argument.

I have had versions of that conversation since with people at other companies, in other domains, arriving at the same wall from different directions. Their agents keep getting better at implementation and their review capacity does not scale with it. The question has changed from how to make agents write more code into how to safely process the code they already produce, and several teams are now working on automated review, risk classification, verification, and ways to cut the low-value work reviewers do by hand. I expect that area to end up as important as the coding agents themselves.

The limits of agent review

There is an appealing answer to all of this: more agents. Agent A implements the feature, agent B reviews it, agent C validates the tests, the pull request merges. There is real value in that model and I use parts of it. An independent agent pass catches bugs, missing tests, inconsistencies, suspicious changes and misread requirements before a human ever opens the diff.

What it does not do is transfer accountability. Two agents can share blind spots. They can misread the same requirement in the same way, and both can produce a convincing explanation of an incorrect implementation, which is a worse failure than a wrong answer delivered badly. Neither of them carries any consequence for deploying the change.

That is not an argument for inspecting every line of every pull request by hand forever. It is an argument that the choice is not between humans reviewing everything and agents reviewing everything, and that picking either end of that gives you a stalled queue or an incident nobody can explain.

Redesigning the review pipeline

The mistake would be leaving today’s pull request process exactly as it is while multiplying implementation throughput by an order of magnitude. That does not scale. The pipeline itself has to get more capable, and the stages it needs are not exotic:

  • Automated verification: build, tests, lint, static analysis, coverage, dependency scanning, generated code checks and basic API compatibility, all of it before a person is involved.
  • Independent review agent: a pass that did not write the code, looking for missing tests, misread requirements and changes nobody asked for.
  • Risk classification: a verdict on what this change can break, which decides the review depth that follows.
  • Human review where it matters: aimed at the decisions that need judgment rather than attention.
  • Merge queue: so the commit that lands is one that was verified against the base it actually lands on.

Each stage exists to reduce the uncertainty that reaches the next one. The purpose of the automation is not to remove engineers from the process. It is to make sure their limited attention is spent where human judgment is the only thing that works.

A narrowing wedge crossed by eight gates, labelled requirement, coding agent, automated verification, independent review agent, risk classification, human review where it matters, merge queue and production
Nothing in this removes the human. It removes the changes that did not need one, so the changes that do get real attention.

A large part of that is deleting work from the human path. A senior engineer should not be spending review time pointing out that a variable wants a better name or that the formatting does not match the project standard. Formatting, linting, static analysis, test execution, coverage checks, dependency scanning, obvious duplication, common error handling mistakes, generated code verification and basic API compatibility checks are all machine work, and every one of them that runs first moves the review toward the questions that need a person. Does this architecture make sense? Did the agent interpret the requirement correctly? Is this abstraction appropriate? Are we introducing hidden coupling? What happens when it fails? Does it violate a domain invariant? Will anyone understand this code in two years?

Smaller pull requests, not larger

Agents are very good at producing large amounts of code, which makes it dangerously easy to produce enormous pull requests. A reviewer handed a 3,000-line agent-generated change has a problem that has little to do with code quality: even when every line is readable, reconstructing the intent, the architecture, the side effects and the interactions costs real effort, and that cost does not scale with the size of the diff in a friendly way.

So agents should be pushed toward smaller, independently reviewable changes. Instead of “implement the new authentication subsystem”, the work becomes a sequence: introduce the new interface, implement token storage, add rotation logic, integrate the authentication flow, add the migration, connect the UI behavior. The agent does not care about the overhead of splitting work across several pull requests. Humans care a great deal about reviewing understandable units of change. Agentic development makes small pull request discipline more valuable, not less.

A pull request should also not reach an engineer as raw agent output. Before it asks for human attention, the producing agent should already have answered the obvious questions: what changed, why it changed, which tests were added, which tests passed, what assumptions were made, which files deserve the most attention, whether any public interface changed, whether database migrations were introduced, whether there are security implications, and what could go wrong. Ideally a second agent has inspected the implementation independently as well. What lands in front of a reviewer should be a pre-processed engineering artifact, not a box with 1,500 changed lines in it.

Not every change carries the same risk

The other assumption that stops holding is that every pull request deserves roughly the same kind of review. Changing documentation, updating generated code, adding a UI component, modifying authentication logic, introducing a database migration, changing authorization boundaries, modifying cryptography and changing concurrency behavior in a distributed system are not equivalent risks, and plenty of processes treat them as though they were.

Agentic development is a good reason to make the classification explicit. Low-risk changes get strong automated verification and light human inspection. Medium-risk changes get one engineer. High-risk changes route to a domain expert automatically. Changes touching authentication, authorization, security boundaries, financial logic, destructive migrations or critical infrastructure integrations require more than one reviewer regardless of who or what produced them. That scales in a way that pretending every changed line deserves identical attention does not.

Three stacked rows labelled low, medium and high risk, each listing the kinds of change it covers on the left and the review those changes require on the right
The examples are mine and the boundaries move per team. The axis is what matters, not my particular list.

Review throughput as a metric

There is an uncomfortable implication in all of this. If implementation gets dramatically faster, teams may have to deliberately stop agents from producing more work.

That sounds absurd until you count unreviewed pull requests as inventory, which is what they are. If ten pull requests are already waiting for review, having an agent generate another ten does not increase delivery. It increases work in progress, and work in progress that nobody has the capacity to finish is only a larger number.

So the limit becomes explicit: no more than three pending pull requests per engineer or agent, for instance. Once an agent hits that ceiling it switches to work that does not add to the queue: improving existing tests, resolving review feedback on what is already open, investigating upcoming tasks, improving documentation, analyzing failures, preparing design proposals. Basic flow management, applied to a world where implementation capacity became very cheap. It also gives you a number worth reporting, which the commit count never was: not how many pull requests were opened, but how many cleared review and how long they waited to do it.

A queue lane between a coding agent and a human reviewer, capped at three pending pull requests by a limit line, with everything past the cap diverted into a list of work that does not add to the queue
Three is an example, not a recommendation. The number worth setting is the one your reviewers actually clear in a week.

What vincent does about this today

This is the problem vincent is aimed at. It is a local, vendor-independent control plane for coding agents: it runs Claude Code, Codex and Cursor as they ship, one task per branch and worktree, through workflows written in validated YAML, and it keeps the state, the gates and the verification on your machine. Most of what it already does is the left half of the pipeline above.

Verification the agent cannot talk its way past. A workflow step can carry a check, a deterministic command that decides whether the attempt actually succeeded. In the feature-pr example workflow that check is check: go build ./... && go test ./..., and an agent that reports success on code which does not build fails there and retries with the real failure appended to its prompt.

Publishing gated on a person. A manual step stops the run and puts rendered instructions in front of a reviewer. Here is the tail of that same example workflow, where nothing leaves the machine until someone approves:

steps:
  - id: commit
    type: command
    run: 'git add -A && git commit -m "{{.Task.Title}}"'

  - id: review
    type: manual
    instructions: |
      Review the diff for task #{{.Task.ID}} on branch {{.Task.BranchName}}.

      The agent ran full-auto: everything it did is in the transcript, and
      `git diff {{.Task.BaseBranch}}...{{.Task.BranchName}}` is the change
      itself. Approve to push, reject to stop here.

  - id: publish
    type: command
    run: git push -u origin {{.Task.BranchName}}

Gates that arrive pre-processed. The workflow vincent uses on its own repository, github-resolve-issue.yaml, has two of them, and their instructions are assembled from what the run actually did. The first gate, gate, authorizes pushing the branch and opening the pull request. It inlines the implementation step’s report and the documentation audit’s report, and its checklist changes with the kind of change: a classify step branches on the issue’s label into bug, ci, enhancement or documentation, so the bug path inlines the reproduction check and asks whether the diff fixes the cause or guards the symptom, while the ci path inlines the commands that were pinned and passed on this host and asks what has not run anywhere else yet. The second gate, gate-merge, spells out what approving authorizes: a rebase, a force-push that rewrites a branch other people may have pulled, and up to two unattended agent repair passes if a required check goes red, which means the diff that merges may not be the diff approved at the first gate. That sentence is in the workflow because a reviewer who does not know it is not approving anything in particular.

The pull request and its checks in the same place. Since 0.9.0 a task’s Pull Request tab shows the pull request’s state next to its CI checks, and merge, close, reopen, comment and re-run are keys on that screen:

The vincent Pull Request tab on a finished task, showing the pull request details, its CI check rows with a failed build selected, and the available key actions
None of these actions is exposed as an MCP tool, so a merge button is not something an agent can reach.

A merge from here targets the exact head whose checks are on screen, and it is refused when the branch is behind, a check is still running, or the head moved.

Splitting the work into units. 0.8.0 taught vincent to resolve one issue as a DAG of parallel lanes, each lane with its own check and its own gate. That is a genuine split of the implementation, and it is not yet a split of the review: the lanes merge back into the parent branch, and one pull request still reaches the reviewer at the end.

The vincent board filtered to a fan-out parent task, expanded to show its storage and client lanes finished and the handlers lane waiting at its gate
Each lane carries its own gate, but they merge into the parent branch rather than opening a pull request each.

Caps on what runs. Global and per-project concurrency limits, priorities, and per-task and per-tree spend ceilings control how much agent work is in flight at once.

What is still open

None of that solves the review side, and I would rather name the gaps than imply the problem is handled.

Risk classification does not exist. classify branches on the kind of issue, not on the blast radius of the diff. Nothing in vincent reads a change, notices that it touched authorization or a destructive migration, and routes it to a different review depth or a different person. That is the piece I most want to build, and the hard part is not the routing. It is deciding what the classifier is allowed to be wrong about.

The independent review agent is not a first-class stage. Vincent ships a cursor-review example workflow that reviews a working tree with a second agent and commits the findings, and any workflow can add an agent step that reviews the previous one’s work. What is missing is the property that makes independent review worth anything: a reviewer whose context, model and prompt are deliberately unlike the implementer’s, so the blind spots do not line up. Running the same model twice mostly buys the same opinion twice.

WIP limits apply to running tasks, not to pending pull requests. Vincent caps how many tasks execute at once. It has no notion of how many pull requests are sitting unreviewed, and no way to tell an agent to stop opening them and go answer review feedback instead. That is precisely the limit the argument above says teams will need, and I have not built it.

I do not know what a human gate should show. The gate instructions in github-resolve-issue.yaml are useful because I hand-wrote them for four specific paths through one repository. That is a workflow author’s craft rather than a product feature, and it does not generalize. What a reviewer needs to see, generated rather than authored, is an open question.

Vincent is pre-1.0, so treat all four of those as a direction I am working in, not a roadmap with dates on it.

A two column comparison. The left column lists the vincent features shipped today with check marks, the right column lists the unsolved problems with hollow markers
The left column runs today. The right column is what I have not solved, the last entry being the fan-out lane limitation above.

Coding faster is not delivering faster

Organizations adopting coding agents measure how much faster implementation became, because that number is easy to obtain and pleasant to look at. Implementation throughput is not delivery throughput. If development gets five times faster while review, testing, security validation, deployment and organizational decision-making stay where they were, the system as a whole does not get five times faster. It accumulates work at the next constraint, and the next constraint is a person.

The same shift makes experienced engineers more valuable rather than less. When code was expensive to produce, implementation skill was the binding constraint. When code gets cheap, the expensive things are judgment, architecture, domain understanding, risk assessment and accountability. An agent will happily generate thousands of lines implementing a particular architecture, and if the architecture is wrong it now generates the wrong system faster than anyone ever has. The role shifts from producing code toward controlling the quality and the direction of an increasingly automated production line.

Agentic coding forces engineering organizations to look at software delivery as a complete system rather than as a coding problem. The teams that get the most out of it will probably not be the ones whose agents generate the most code. They will be the ones that redesigned their process around a cheap supply of code and a fixed supply of attention. Code is getting cheap. Judgment did not, and trust is on its way to being the most expensive resource in the whole lifecycle.

What I want to hear from you

I am writing this partly to think it through and mostly because I want other people’s data. If you are living this right now, these are the things I would find genuinely useful:

  • Review load: how many agent-generated pull requests a week your team actually absorbs, and how many people review them.
  • Risk classification: whether you classify changes before review, what the axes are (change type, touched component, blast radius, something else), and who or what decides the class.
  • Agent review you trust: whether anyone has made automated review good enough to reduce human review rather than add a step to it, and what evidence convinced you it was safe to lean on.
  • WIP limits: whether you cap pending pull requests per engineer or per agent, what the number is, and what the agents do once they hit it.
  • The gate itself: what you put in front of a reviewer before they open the diff, and which parts of it you have watched them skip.

Negative results are as useful to me as the successes. If you tried agent-approved merges and pulled the policy back, I want to know what made you stop. Open an issue on vincent’s tracker if it is something the tool should do, or write to me at lezli01@gmail.com if it is a story rather than a feature. What I learn goes into vincent and into a follow-up post.

The documentation and the repository cover what exists today, and the rest of what I build is collected at lezli01.is-a.dev.

← all posts