A growing share of the code in any given repository was never typed by a person. It was generated, accepted, and shipped, often in a pull request too large for anyone to read line by line before merge. That is not a productivity footnote. It is a change to the attack surface, and most security programs are still scoping their reviews as if every line had a human author who at least glanced at OWASP once.
The uncomfortable part is not that AI assistants write insecure code sometimes. It is that they are optimized to write code that runs, and running is a much lower bar than safe. A model trained to satisfy a prompt will produce an endpoint that returns the right JSON while skipping the authorization check nobody asked for by name. It will also happily suggest a package that sounds exactly right and does not exist, because plausibility and reality are not the same objective function.
Why this is hard now, specifically
The volume problem is real and it is new. When a single engineer can generate a thousand-line feature branch in an afternoon, the review that used to take an hour now needs to cover ten times the surface in the same window, or it gets rubber-stamped. Teams that measure velocity by commits merged or PRs closed are quietly rewarding the rubber stamp, because the alternative is missing sprint targets.
Static analysis, which used to be the safety net under a fast-moving team, was tuned for how humans write bugs. Humans repeat a handful of mistakes across a codebase, which is exactly the pattern SAST rulesets are built to catch. AI-generated code produces a wider, less consistent spread of small mistakes, one prompt at a time, and a lot of it looks locally fine while being wrong about intent. A query that is technically parameterized can still be an authorization bypass if the model never modeled who is allowed to see the row.
The dependency layer adds a second, separate problem. Large language models trained on package ecosystems occasionally hallucinate import names that read as plausible but resolve to nothing. Academic and vendor research on this behavior, including a 2026 Cloud Security Alliance research note and reporting from the supply-chain security vendor Socket, has measured meaningful rates of these fabricated package recommendations, and found that the same fake names recur across repeated generations rather than being one-off noise. That consistency is what makes the risk exploitable rather than merely embarrassing.
What actually changes
Three things change in practice, and they map to three different disciplines.
In a penetration test, scope now has to ask where the code came from, not just what the code does. A test plan built around OWASP Top 10 categories still applies, but the tester needs to weight effort toward business logic and authorization flaws over injection, because that is where AI-assisted development research consistently finds the gap. Vendor testing across a large sample of models, published by Veracode, found a substantial share of AI-generated code samples introduced an OWASP Top 10 class vulnerability on security-sensitive tasks, with wide variance by language. That is a reason to ask, before the engagement starts, which parts of the target were AI-assisted and treat those paths as higher prior probability of a finding, not a reason to skip them.
In application security review, the dependency manifest becomes as interesting as the code itself. A pull request that adds a new package your team has never heard of used to get a second look because it was unusual. Now it is common, because an assistant suggested it, and the question is whether that package exists, is maintained, and is the package the developer thinks it is. That means pinning to verified hashes, checking a package's registration date and download history before it enters a lockfile, and treating a first-time dependency the same way you would treat a first-time vendor, with due diligence before it gets access to anything.
In secure development practice, the review gate has to move earlier and get narrower per change, not later and broader. A thousand-line AI-generated PR reviewed as one unit gets a shallow pass. The same logic broken into smaller, single-purpose changes gets a real one. That is a workflow change, not a tooling purchase, and it is the one most teams resist because it looks like it slows delivery down. It does not slow delivery down as much as an unreviewed authorization bug does six months later.
An AI assistant asked to add a logging utility to a Python project suggests a package. It sounds exactly like a real, obscure but legitimate library, matching the naming convention of packages that do exist. The developer installs it without checking, because the name resolved cleanly in the assistant's chat window and the assistant said it was the right tool for the job. The package was never registered on the public index before that moment. Whether it resolves to nothing, to an abandoned name someone else grabbed, or to a package an attacker registered after watching for exactly this pattern, depends entirely on who got there first.
What this looks like in a real engagement
Scoping conversations now include a direct question early on. What fraction of this codebase, and specifically which services, were built primarily through AI-assisted generation, and over what time window. That answer changes where a tester spends hours. A service built in the last two quarters through heavy AI assistance gets more time on authorization logic, session handling, and dependency review, and less time on classic injection sweeps that automated tooling already covers well.
Dependency review stops being a once-a-quarter software composition scan and becomes a pull request gate. Every new package, whether a human typed the import or an assistant suggested it, gets checked against the actual registry before the lockfile update merges. This is a process change your engineering team owns, not a control a pentest can retrofit after the fact.
Code review practice shifts from reading for style to reading for intent. The question stops being does this pass linting and becomes does this do what the ticket asked, and nothing more. That is a harder review to do and a much more valuable one, because it is the review that catches the endpoint that works correctly and also quietly skips a permission check.
The sharpest version of this risk is not a single bad dependency. It is a lockfile that already contains one, merged weeks or months ago, sitting quietly with no clean signal to distinguish it from every other legitimate transitive dependency in a tree hundreds of packages deep. Retroactive discovery depends on registry-level detection and dependency provenance checks that most teams have never run, because nothing in a normal build failed. A build that succeeds is not the same thing as a build that is safe.
Where teams get this wrong
The most common mistake is treating this as a tooling gap and buying a scanner instead of changing a workflow. A scanner that flags a nonexistent package after it is already in production caught the problem after the point where it mattered. The second mistake is assuming AI-generated code is uniformly worse and applying blanket suspicion, which burns review budget on low-risk boilerplate while the actual authorization flaw sits in a PR nobody flagged as AI-assisted because nobody asked. The third is measuring AI adoption by lines shipped per week, which is precisely the metric that rewards the giant, unreviewable pull request.
Where this leaves you
None of this means AI-assisted development is a mistake. It means the assumptions your review process was built on, that code volume tracks roughly with human attention span and that a package name in an import statement is trustworthy by default, no longer hold. A penetration test that does not account for how a codebase was built is testing last year's threat model against this year's code. Scope for what changed.
Related Service
Learn more about how we can help with Penetration Testing.
Explore Penetration Testing Services →
