Vercel's Agents Write 25–35% of Merged PRs. Human Approval Is the Point.

Vercel says agents in its AI SDK software factory now author 25–35% of the pull requests it merges each week. A human on the AI SDK team still approves every merge.

The first number is the one people will repeat. The second is the one worth copying.

Four weeks into production, Vercel also reports that the factory closes 70–80% of issues. In July, it handled more than 75% of closed issues, while the open backlog fell from 1,022 in late June to 844 by early August. Those are meaningful results. But they do not come from giving a coding agent a backlog and asking it to ship whatever it can.

The factory does something more disciplined: it turns a request into a reviewable chain of evidence before a human makes the final decision.

My take is simple: scale evidence before authority.

The percentage is not the product

“Agents wrote a third of our merged pull requests” sounds like an autonomy story. Read the workflow and it becomes a review-system story.

The AI SDK factory uses specialized agents for classification, analysis, implementation, review, backports, documentation, and other narrow jobs. Each step produces an artifact that the next step—or the human reviewer—can inspect.

For a feature request, that can mean:

1. Classify the request and explain why it belongs in the project.
2. Reproduce the missing behavior or confirm the problem.
3. Write a concrete implementation spec.
4. Make the code change.
5. Run a real test and attach the result.
6. Assess side effects, performance, and compatibility risk.
7. Send the code and evidence to a human for approval.

That is not a model generating more code. It is a system reducing the amount of uncertainty a reviewer has to resolve.

This distinction matters because code generation is already cheap. Review attention is not. If agents increase pull-request volume without making each change easier to judge, they only move the bottleneck downstream.

Human review scales with risk

Vercel does not apply the same review depth to every change. A documentation fix can get a quick verification. A provider change needs focused validation. A new public API gets deep review.

That sounds obvious, but many agent workflows flatten risk. Every task gets the same prompt, the same checks, and the same approval ritual. Low-risk work becomes unnecessarily slow while high-risk work receives a review process that is too shallow.

The better pattern is to make risk explicit before review begins. The agent should not only propose a change. It should tell the reviewer what could go wrong and show the evidence behind its confidence.

The human then spends judgment where judgment is most valuable.

This is why “human in the loop” is not enough as a policy. It says who clicks the button, but nothing about what they see before clicking it. A tired reviewer approving a poorly explained change is technically in the loop and practically operating blind.

What the evidence chain looks like

Vercel’s example starts with a community request for blocked-domain support in OpenAI web search. The factory classified it as a feature, then wrote and ran a probe confirming that the capability was missing from the SDK.

The analysis agent turned that result into a spec. The implementation agent made the change and ran a live end-to-end test with Wikipedia blocked. A review agent assessed implementation completeness and rated side-effect, performance, and backward-compatibility risk. Lars Grammel then read the evidence, reviewed the code, and merged the pull request.

The factory later opened backports for two older SDK versions. When one backport hit a conflict, an agent preserved the conflicted state, found a fix, validated it, and pushed the correction for review.

The useful part is not that an agent touched every stage. It is that every stage left something inspectable behind.

The reviewer did not receive “done.” The reviewer received a claim, a reproduction, an implementation, test output, a risk assessment, and known boundaries.

A minimum contract for agent-created pull requests

You do not need Vercel’s infrastructure to use this pattern. Start by defining what an agent-generated pull request must contain before a person reviews it.

I would require four things:

1. Reproduction or analysis

Show that the bug exists or that the requested behavior is actually missing. For a feature, explain why it fits the product and the surrounding architecture. Do not let the implementation be the first evidence that the task was understood.

2. Test evidence

List the exact checks that ran and their results. Include the failing state before the fix when possible, then the passing state after it. “Tests pass” is a conclusion. The commands and output are the evidence.

3. Risk assessment

State what could break: public APIs, compatibility, performance, security, data, or deployment behavior. Match the depth of human review to that risk instead of treating every diff as equal.

4. Known limitations

Say what was not verified. Name missing credentials, unavailable services, untested platforms, or assumptions the agent could not confirm. A clear boundary is more trustworthy than false completeness.

None of these removes the need for a reviewer. They make the review worth the reviewer’s time.

More proof, not less accountability

Vercel’s factory is still early, and the reported results belong to Vercel’s codebase, team, and operating environment. They are not a promise that another team will automate the same percentage of work.

The transferable lesson is the shape of the system.

Give agents responsibility for collecting proof before giving them more authority. Make low-risk changes quick to verify and high-risk changes hard to approve casually. Treat flawed, blocked, and manual runs as feedback for improving the factory rather than reasons to hide uncertainty.

The best coding-agent workflow is not the one that produces the most pull requests. It is the one that makes a good human decision easier.

What would you add to the evidence contract?

Source: Vercel, “Building a software factory for AI SDK”