SQL Injection and XSS in AI-Generated Code: Why LLMs Keep Failing

Veracode tested more than 100 LLMs on 80 coding tasks specifically designed to give models a choice between secure and insecure implementations, and the results are hard to dismiss: AI-generated code introduced security flaws in 45% of tests, with XSS failing at an 86% rate. The default narrative frames this as a "model knowledge gap" that human review is supposed to catch. That framing is wrong, and believing it leads to security processes that don't actually reduce risk. The real pattern is that LLMs don't fail randomly on SQL injection and XSS. They fail systematically, and the failure mode is more closely tied to how the prompt is structured than to what the model "knows."

XSS Fails 86% of the Time Because the Prompt Doesn't Give the Model Enough Context

The Veracode finding that models fail to generate secure XSS-resistant code 86% of the time deserves more attention than it usually gets, because the explanation points directly at prompt structure rather than model capability.

XSS vulnerabilities require output encoding, and output encoding requires knowing the context: are you inserting into an HTML attribute, a script block, a URL parameter, or raw HTML content? Each context demands a different escaping strategy. When a prompt asks for a "user profile page that displays the username," the model doesn't know which rendering context it's operating in, so it defaults to what's most common in its training data, which is unescaped string interpolation.

This is testable. Give the same model a prompt that specifies the rendering context explicitly and asks for secure output handling, and the failure rate drops. The model knows what DOMPurify is. It knows what htmlspecialchars() does. The knowledge exists; the prompt didn't activate it.

The deeper issue is that training corpora skew toward functional examples. A Stack Overflow answer that shows how to display user input on a page is optimized to answer the narrow question being asked, not to model a complete security posture. The Veracode 2025 GenAI Code Security Report attributes the failure pattern partly to this: "codebases included in training datasets are more likely to be functional than secure, and are unlikely to be labeled as secure or insecure before being ingested."

In Java specifically, the failure rate climbs to 72%. Veracode's Wessling explains the mechanism directly: Java predates the formal recognition of SQL injection as a vulnerability class, so the volume of legacy, insecure Java examples in training data is disproportionately high. The model learned from what was available, and what was available was mostly written before secure-by-default was a design goal.

The Vulnerability Pattern Is in the Prompt, Not the Model's Knowledge

This is where the generic advice breaks down. "Always review AI output" treats the model as a junior developer whose code needs inspection before shipping. That's not wrong as a practice, but it misidentifies the source of the problem and therefore produces the wrong fix.

The more accurate framing: LLM security failures on SQL injection and XSS are prompt-deterministic. The same model that produces a raw string-concatenated SQL query when asked to "write a login function" will produce a parameterized query when asked to "write a secure login function using prepared statements." The model's behavior changes based on what the prompt signals it should optimize for.

This pattern is consistent with controlled testing on adversarial prompting of code models, and it aligns with what Veracode's methodology shows: models have measurable security capability that doesn't activate under typical prompting conditions. The failure is that the prompt framed the task as "get this working" rather than "get this working securely," and the model satisfied the stated objective. The model isn't unaware that $_GET['id'] appended directly into a SQL string is dangerous; it just wasn't asked to care about that.

The practical implication is that security reviews positioned at the end of the pipeline, as a catch-all for AI output, will always lag behind the volume of code being generated. Apiiro's September 2025 research found that AI-assisted commits were merged into production 4x faster than regular commits, meaning insecure code bypasses normal review cycles before a human ever sees it. By June 2025, AI-generated code was introducing over 10,000 new security findings per month across the repositories in their study. A 10x spike in six months with an accelerating curve is not a problem that end-of-pipeline review is designed to handle at that volume.

The fix that actually addresses the root cause is moving security specification into the prompt itself. This means:

• Naming the security requirement explicitly ("use parameterized queries", "apply context-aware output encoding")
• Specifying the rendering or execution context so the model can choose the correct defense
• Framing the task around the secure implementation rather than the functional one

This isn't a guarantee of safe output. It's a structural change that shifts the model's optimization target at generation time, which is the point where the problem actually originates.

The Architectural Flaw Problem That SQLi and XSS Coverage Misses

Fixing individual vulnerability classes in prompts does not solve the larger pattern that Apiiro's data surfaces. Syntax errors in AI-written code dropped 76% and logic bugs fell by more than 60%, which is the improvement teams usually cite when arguing that AI assistants are net positive for code quality. Those numbers are real. The tradeoff is the part that gets less attention: privilege escalation paths jumped 322%, and architectural design flaws increased 153% in the same dataset.

SQL injection and XSS are shallow vulnerabilities in the sense that they're detectable at the function level. A single parameterized query replaces a single interpolated string. The architectural flaws Apiiro is tracking, things like overly permissive access control, insecure service-to-service trust relationships, and missing authorization layers, don't resolve with a prompt instruction because they emerge from how components relate to each other, not from how any single function is written.

This matters for how you read the XSS and SQLi numbers. An 80% pass rate on SQL injection looks like progress until you recognize that the 20% failure rate still represents a significant volume at scale, and that the same code generation workflow is simultaneously producing privilege escalation paths at three times the previous rate. Fixing the shallow class doesn't neutralize the architectural one. If anything, the speed improvement that AI assistants provide (the 4x faster merge rate Apiiro documents) means both categories reach production faster than before.

Itay Nussbaum from Apiiro put it bluntly in The Register: "AI is multiplying not one kind of vulnerability, but all of them at once." The corollary for teams using AI-assisted development is that prompt-level security specifications address one layer of the problem, and dedicated application security tooling has to address the rest in parallel, not as a downstream catch.

If you're already thinking about how prompt structure relates to what the model produces at a system level, the mechanics here connect directly to what's covered in red-teaming system prompts against injection before ship, and the training-data contamination angle overlaps with the patterns documented in hardcoded secrets in AI-assisted codebases.

From My Experience

In my work managing product catalogs and marketplace integrations, I don't write production code, but I do regularly use AI tools to generate scripts for data transformation, feed validation, and GA4 event mapping. The pattern I've noticed, without any controlled testing, is that the output quality on security-adjacent concerns like data sanitization or input validation tracks almost directly with how explicitly I name the requirement in the prompt. When I describe what I want the script to do, I get something functional. When I add a line about how untrusted input should be handled, the output changes meaningfully. Whether that delta translates to actual security is for someone with AppSec expertise to evaluate, but the behavioral difference in the model is visible and consistent enough that it stopped feeling like coincidence after the first few iterations.

FAQ

Does using a more capable model (GPT-4o, Claude 3.5 Sonnet, etc.) reduce SQL injection and XSS rates significantly?

The Veracode report tested over 100 models released between March 2023 and May 2025, and while newer models show improvement on some vulnerability types, no model in the study achieved consistently secure output without explicit security framing in the prompt. Capability level helps at the margins but doesn't eliminate the prompt-dependency pattern. The full report breaks down results by model generation.

Is static analysis enough to catch AI-generated vulnerabilities before they ship?

For shallow vulnerabilities like SQL injection and XSS, SAST tools catch a reasonable percentage. For the architectural flaws Apiiro documents (privilege escalation, insecure service trust), static analysis alone is insufficient because those issues emerge from cross-component relationships that single-file analysis doesn't model well. The answer for most teams is SAST plus DAST plus manual review on security-critical paths, not any single tool in isolation.

Why does Java perform so much worse than Python or JavaScript in Veracode's tests?

Veracode's explanation is training data composition. Java as a language predates the formal classification of SQL injection as a vulnerability, so the volume of legacy, pre-security-awareness Java examples in public repositories is disproportionately large. Models trained on that corpus learn patterns from code written before secure-by-default was a design norm. Python and JavaScript have lower failure rates, though 38% and 43% respectively are not numbers anyone should find reassuring.

The cleanest takeaway from Veracode's controlled methodology is that the same model passes explicit security tasks and fails implicit ones, which means the accountability for AI-generated vulnerability patterns sits in the prompting workflow, not exclusively in the model's training. That's a more actionable insight than "review your AI output," because it identifies where in the process the intervention actually needs to happen.