AI will find you an endless supply of problems. Catching the ones it hands back is the new job.
Last month I pointed an agent at a security backlog: a long list of overdue items across a dozen repos. Dependency alerts, base-image CVEs, a few real vulnerabilities. This kind of list sits untouched because nobody has a free afternoon and half the repos belong to people who moved on.
The agent went through all of it in one pass. It reduced the list to a handful of real pieces of work. It caught that two of our scanners disagreed: one had flagged serious issues the other missed. It told me the automated PRs were never coming, so there was no point waiting for them. It opened the safe fixes as draft PRs, one repo at a time, with tests green.
Then it did the interesting thing. A few findings were real vulnerabilities. They needed code changes and a judgment call about how bad they were. For those, the agent wrote a remediation plan and stopped. It handed them back and did not auto-fix. It also flagged one fix that was blocked only on a human review approval. It said so plainly: this needs a human, not code.
That last move is the whole story. The agent found everything and fixed the boring parts. As soon as a finding needed an owner, someone to accept the risk and put a name on the merge, it handed the potato back to me.
Finding is cheap, owning is not
We spent years treating “find the problem” as the hard part. Static analysis, scanners, linters, test suites and security tools all exist to surface issues, because surfacing issues used to be slow and expensive.
AI changed that. Finding is now close to free. An agent will read a codebase it has never seen, cross-check two scanners, dedupe a backlog, and spot what your linter missed, overnight. It is very good at this.
It also got good at the mechanical half of fixing. Bump the dependency. Rebase the branch. Write the test. Open the PR. This work is tedious, not risky.
What it hands back, correctly, is judgment and accountability. Is this vulnerability exploitable in our setup, or only on paper? Do we ship the fix now or accept the risk until next sprint? Who signs off that this merge won’t take down a customer? These are ownership questions, and an agent can’t own anything. It carries no pager and faces no consequences.
So the potato comes back to a human. Every time.
More findings arrive faster
This is the part people miss when they get excited about AI review. When finding is free, the backlog gets louder.
Point a naive agent at a diff, tell it to find problems, and it will. You get a flood of maybe-bugs, style nits, invented concerns, and “consider adding error handling” on a function that already handles errors. The good review systems spend most of their engineering on the opposite job. A coordinator dedupes the findings, judges which are real, drops the noise, and decides what deserves a human’s attention. The most effective prompt instruction in those systems tells the model what not to flag.
So we built tools that find issues for free, then had to build a second layer whose only job is to decide which of those issues matter. The judgment did not go away. It moved up a level.
Now spread that across a team that is already overloaded. Every agent surfaces more findings than the humans have context to resolve. The repos nobody remembers still generate alerts. Reviews slow down because assessing an unfamiliar fix takes as long as writing it. The agent keeps handing back potatoes, and there aren’t enough hands to catch them. So people pass them sideways: to another person, another team, a backlog labeled “later.” Nobody drops them. They circulate.
That circulation is the work now. The finding is the easy part.
Don’t stop at the easy part
Finding is the half we’ve gotten comfortable with. It demos well and it feels safe, because a finding is only information. You read it and move on. So most teams point AI at their code, collect a list of problems, and stop. They use a frontier model as a smarter linter.
The same model that spotted the bug can usually propose the fix, write it, test it, and give you a diff to read. That is the higher-value half, and we use it least. Finding a problem and doing nothing about it is the cheapest outcome there is. If the agent is good enough to find the issue, have it work on the fix too.
There’s a double standard here. We trust the agent to find problems. We act on what it finds. We reorganize the week around its list. Then we don’t trust it to fix those same problems. That is backwards. Finding the right problem is usually the harder call. Deciding that a vulnerability matters, that a design is wrong, or that this is the root cause and not a symptom takes judgment. Bumping the version once you know is mechanical. We trust the model on the hard half and supervise it closely on the easy half.
The obvious objection is that fixing is more complex than finding. True. That argues for keeping a human on the diff. It does not argue for refusing the fix. We already read the fix line by line, while we act on the finding without re-deriving it. The harder task already gets more scrutiny.
Some hand-backs are justified. Irreversible changes, security calls, and anything with a wide blast radius should come back to a human. Take those out, and the rest usually has one of two causes: it feels safer, or you lack the guardrails to accept changes at this pace. The second cause is the honest one. AI did not break your confidence. It showed that your tests, CI, review and rollback were never strong enough to let you say yes fast. Where the guardrails are good, you accept the fix without much thought. Where they are missing, accepting it feels reckless, and the missing guardrails are the reckless part. A change that is small, tested, reversible and behind a flag costs almost nothing to accept. A change that isn’t shows you what to build next.
What this changes if you run a team
Stop being impressed by finding throughput. An agent that opens 200 findings is not ten times more useful than one that opens 20. Findings are cheap. The scarce resource is the capacity to own the resolution: the context to judge it and a person willing to put their name on the call.
A few things I’m trying to hold onto.
Measure what got owned, not what got found. Count the issues that got resolved, merged, and closed by someone who understood them. The number of issues the tools surfaced is now a vanity metric.
Build the catching layer on purpose. The finding agent is the cheap part. The value is the routing and judgment around it: what is real, what matters, who owns it, and when the agent may auto-fix instead of handing back. The better internal reviewers I’ve seen build this in. They use a strict “hand back to a human” rule for anything serious, plus an override so a person can always take control. Let the agent fix what is safe and reversible. Route everything with real consequences to a named human.
Watch where the potato lands. When the same unfamiliar repo keeps generating findings nobody can own, you have an ownership problem, and the AI made it visible. The fix is to settle who owns what. A better scanner won’t help. Sometimes the most useful thing an agent does is show you which parts of your org have no owner.
Keep humans on the hand-back items on purpose. Risk acceptance, security calls, cross-cutting merges. These should come back to a person, because someone has to be accountable. You can’t give accountability to something with no stake in the outcome.
The skill that used to matter was finding the problem. That skill is now a commodity. The skill that matters now is catching what gets handed to you, fast enough and with enough judgment that the pile goes down.
AI will find you an endless supply of issues. It will fix the easy ones and hand the rest back with a tidy summary. It will never decide which ones are worth your afternoon, or put its name on the merge.
That’s still the job.