I write code with AI every working day. Claude Code sits in my editor, and a decent share of what ships in my bespoke builds started as a model’s suggestion that I read, argued with, and rewrote. So when 1Password’s new research team published a study on AI security patching, I paid attention: one of the two tools they tested is sitting on my desk.
The short version: left to work on its own, a frontier model produces a clean, correct security fix roughly one time in four. The longer version is more interesting, and rather less flattering to everyone involved, including the humans.
First, what a patch has to do
A vulnerability is a mistake in code that lets somebody do something they shouldn’t: read data they don’t own, run commands on a machine that isn’t theirs, crash a service on demand. A patch is the change that closes it.
That sounds like a small, contained job, and this is where most people’s intuition goes wrong. The same mistake usually exists in more than one place. Code gets copied, similar functions get written by different people at different times, and a single logical flaw ends up reachable by four or five different routes through the program. Closing the route that somebody demonstrated to you is not the same as fixing the flaw. It’s locking the door that the burglar used and leaving the other three open, and that distinction is where the whole problem lives.
What the study did
The team (Off-by-1 Labs, 1Password’s research group) picked six serious, recently disclosed vulnerabilities in well-known open-source software: the Linux kernel, Google Chrome, the Exim mail server, Apache ActiveMQ, Spring AI, and Google’s Gemini command line tool. The timing matters: because these bugs were disclosed so recently, the models almost certainly hadn’t seen the solutions in their training data.
Each of these had already been fixed properly by the people who maintain the software, so there was a marked answer sheet to compare against. The models were never shown it.
They then ran the two most widely used coding models, from Anthropic and OpenAI, at the problem: over six thousand separate attempts in total. They varied how the model was set up (one single shot with no ability to test its work, versus loops where it could run a test and try again) and how the problem was described to it (a terse bug report, a formal security advisory, a casual message from a colleague, an automated scanner’s finding, and so on). Every resulting patch was then graded.
The headline numbers
Across all six thousand attempts:
- 26% closed the vulnerability cleanly, with no side effects.
- 20% closed it, but changed how the software behaved in ways the real fix didn’t; working code that now rejects things it should accept, for example.
- 49% left at least one route to the vulnerability open.
- Around 4.5% introduced a brand new security hole in the process of patching the old one.
Of the patches that were graded as successful, 37.5% were flagged fragile: the specific attack was blocked, but the underlying broken machinery was still sitting there, waiting for some future bit of code to call it from a different direction.
The finding I’d write on a Post-it note stuck to my monitor
Buried in the middle of the report is the result that changed how I write prompts.
The researchers varied not just how much information they gave the model, but whether that information was correct. Sometimes they handed over an accurate diagnosis of the root cause. Sometimes they handed over a plausible-sounding but wrong one, of the sort an automated scanner or a rushed colleague might produce. Sometimes they gave nothing but the symptom.
The results:
- Correct guidance: 65% success.
- No guidance at all: 50% success.
- Confidently wrong guidance: 15% success.
Being told the right answer helps a bit. Being told the wrong answer is catastrophic, and worse than being told nothing. A wrong theory costs you three times more than a right one gains you.
Worse still, the researchers watched models go and investigate the code, find evidence that flatly contradicted the theory they’d been handed, and then implement the theory anyway. Picture a mechanic told the problem is in the engine. He walks round the car, clocks that one of the wheels is missing, and starts pulling the engine apart.
Human developers do get led astray by bad bug reports. But a person has to reason their way to a fix, and that reasoning tends to surface the contradiction. A model running unattended, with no conversation and nobody pushing back, mostly just does as it’s told.
Tunnel vision
The second recurring failure is the one I flagged at the top: fixing the door rather than the lock.
When a security bug is reported, it usually comes with a reproducer, a small script that demonstrates the problem by triggering it. It’s proof the bug is real, and it’s how you check your fix worked. A human reads it as one example of a general problem. The models read it as the problem.
Give a model a reproducer and it will reliably patch precisely the path that reproducer exercises, then declare victory, while an identical copy of the same flaw sits three functions away, untouched. The test goes green. The vulnerability is still there.
In the Chrome case, the correct fix required protecting two related pointers. The models found and protected the first one around 60% of the time, and of those, roughly 40% never touched the second. The bug wasn’t fixed so much as relocated.
Let’s not get carried away
The report is careful and honest about its own limits, which is more than you can say for a lot of AI research. Three things temper the headline.
The bugs were chosen to be brutal. Complex, multi-route flaws in enormous codebases maintained by some of the best engineers in the industry. That’s the hardest few percent of security work, not the median. The study answers “can AI do this unsupervised on the hard stuff”, which is a fair and important question, but it isn’t the same as “is AI bad at fixing security bugs”.
The marking was mostly done by AI too. Six thousand patches is more than a small team can review by hand, so the grading was automated, with human spot checks on a sample. When they compared the automated grades to their own manual review, the grades matched on the full verdict about two thirds of the time. The broad trends will survive that noise. The precise percentages should be held loosely.
The answer sheet had errors in it. In two of their examples, the official fix written by human maintainers and accepted into the codebase was itself defective.
In the Linux case, the maintainers’ fix introduced an off-by-one error: a counting mistake where a loop runs one step further than it should, writing into memory that belongs to something else. Exactly the class of bug they were trying to fix. They caught it and corrected it a commit later. Then the AI models, working independently and without access to that fix, generated the same flawed pattern in around a third of their attempts.
The second example is worse. A use-after-free bug in a web server had two competing human fixes: one submitted by a well-regarded security consultancy as part of an AI-assisted patching initiative, one written by the maintainer. The consultancy’s was rejected. The maintainer’s was accepted and shipped. Under the study’s own grading system, both were flawed. The submitted one didn’t close the whole bug and added a new crash. The accepted one closed the bug and added the same crash. When the researchers then ran 270 AI attempts at that same vulnerability, not one produced a clean fix; every attempt that closed the original hole inherited the same new problem the humans had.
Which brings me to the question the report doesn’t ask. It concludes that the expected value of an unreviewed AI patch is negative. Negative against what, though? The same six bugs were never put in front of human engineers, so there’s no control group to weigh it against. The evidence sitting inside the report suggests that unsupervised patching of nasty, multi-route memory bugs has a high failure rate for everybody, and the real question isn’t “is the AI bad at this?” but “is it worse, by how much, and at what cost?”
What I take from it
I don’t read this as a reason to stop using AI to write and fix code. I read it as a fairly precise description of where the danger sits, and it maps onto four rules I already work by.
A person stays in the driving seat. The failure modes in this study are failure modes of automation, not of the model. Every one of them (following a wrong theory off a cliff, patching the demonstrated route only, calling it done when the test goes green) is the sort of thing that dies in about thirty seconds of a competent human reading the change. The value is in the assistance, and it evaporates the moment nobody’s reading the output.
Describe the symptom, not your theory. If I’m confident about the root cause, saying so helps. If I’m guessing, my guess is worse than silence. I now hold back hunches I’m not sure of and let the model go and look, then argue with what it finds.
A passing test proves the test passes. It doesn’t prove the bug is gone. It proves the route you thought of is closed. This applies well beyond security, and it’s the reason I’d rather fix a known bug and re-run the existing suite before adding anything new, rather than the other way round.
Ask suppliers what the review step looks like. If a vendor tells you their security patching is automated, the number that matters isn’t how fast they ship fixes. It’s who reads the diff before it goes out, and what they’re qualified to spot. “We patch within four hours” is not reassuring if a quarter of those patches work.
But what if you can’t read the code?
Every one of those rules assumes there’s somebody in the room who can look at a change and tell whether it’s right. That’s the safeguard. It’s the only one on the list, and the other three are just ways of making its job easier.
Plenty of people building software with AI right now don’t have that person, and know it. Someone with no development background has spun up a working app in Lovable or Replit or Power Apps, it does the thing, and they’re pleased with it. Fair enough; I’d rather they built it than didn’t.
The report’s findings land harder on that group, though. A model handed a wrong theory runs with it and won’t argue. If you can’t read code, you can’t supply a right theory and you can’t spot a wrong one, which puts you in that 15% bucket by default every time something breaks. And the failure that bites won’t be the dramatic one. It’ll be the 20% category: a fix that solves the reported problem and quietly changes something else. That’s invisible unless you already knew what the behaviour was supposed to be.
Research cited in the same report is blunter still about AI-written code in general. Veracode found close to half of it contains known vulnerabilities when nobody supplies security guidance. The Cloud Security Alliance found AI-assisted developers shipping code three to four times faster than their peers while introducing security findings at ten times the rate. Faster, and disproportionately wrong.
None of that means don’t build. It means the question isn’t how complicated your app is, it’s what happens when it’s wrong.
If the answer is “I lose an afternoon and do it again”, crack on. Internal tools, prototypes, dashboards over data you already have, automations that kill a manual job: this is what these tools are good at, the failure mode is a broken tool, and you’ll notice.
If the answer involves somebody else’s money, somebody else’s personal data, or somebody else’s safety, you need a reviewer, and no amount of careful prompting is a substitute. In practice:
- Payments. Don’t build it. Use hosted checkout so card details never touch your code at all. Where there’s nothing to steal, there’s nothing to get wrong.
- Logins and permissions. Use an established provider rather than rolling your own. Getting “who is allowed to see what” subtly wrong is the most common way an app leaks data, and it’s the sort of wrong that behaves perfectly in testing.
- Personal data. UK GDPR obligations sit with you, not with the tool that wrote the code. “The AI built it” is not a defence and won’t be treated as one.
- Anything a business depends on staying up.
The pattern is the same throughout: keep the security-critical parts out of your own code entirely and hand them to something built by people who do this full time, then get one review before it goes live. A few hours of somebody competent reading it costs a great deal less than the alternative, and less than trying to learn application security for the sake of one project.
I’ll keep using these tools, because they save me real time on real client work. I would not let one close a security ticket without reading every line of what it changed, and after reading this, I’d have a harder look at the lines it didn’t.
The full report, Frontier Models’ Vulnerability Patches are Often F.L.A.W.E.D., is published by Off-by-1 Labs at 1Password and is available here. The testing toolchain has been released publicly, so you can run it against your own codebase, which is the recommendation I’d endorse most strongly out of everything in it.