The Code Works. The Security Does Not.
AI code assistants have changed how software gets built. Over 70% of professional developers now use AI-assisted coding tools according to the 2024 Stack Overflow Developer Survey. The code these tools produce compiles, passes tests and ships to production. But it carries a class of security vulnerabilities that are distinct from what human developers typically write.
We audit AI-generated codebases regularly at Sherlock Forensics. These are the five vulnerability patterns we find most often.
1. Hallucinated Package Dependencies
This is the vulnerability unique to AI-generated code. When you ask an AI assistant to implement a feature, it sometimes references packages that do not exist. The AI generates an import statement for a plausible-sounding library name that it synthesized from training data patterns. The package has never been published to npm, PyPI or any other registry.
Researchers at Vulcan Cyber documented this in 2023 when they found that ChatGPT hallucinated package names in over 30% of code generation tasks. Attackers have taken notice. The attack is straightforward: monitor AI outputs for hallucinated package names, register those names on public registries, then wait for developers to run npm install or pip install.
In one audit we found a Node.js application importing express-auth-helper, a package that did not exist on npm at the time. The developer had copied the AI output verbatim. Had an attacker registered that name first, every build would have pulled in malicious code.
Detection: Verify every import statement against the live package registry. Automated dependency resolution catches this at install time but not at code review time.
2. Weak Randomness in Security Functions
AI assistants default to the simplest implementation. When generating token creation, session management or API key generation, they routinely use insecure random number generators.
In JavaScript, Math.random() appears instead of crypto.getRandomValues(). In Python, random.randint() appears instead of secrets.token_hex(). In Java, java.util.Random appears instead of java.security.SecureRandom.
The output looks random to the developer. It is predictable to an attacker who can observe a few outputs and reconstruct the PRNG state. We have exploited this in penetration tests to predict password reset tokens, session IDs and API keys generated by AI-written code.
Detection: Grep your codebase for Math.random, random.randint and java.util.Random in any security-relevant context. Replace with cryptographically secure alternatives.
3. SQL Injection from String Concatenation
This one is almost comically consistent. Ask an AI assistant to write a database query that uses user input and it will generate string-concatenated SQL a significant percentage of the time.
The AI knows what you want to query. It builds the correct logic. But it constructs the query by inserting user input directly into the SQL string instead of using parameterized queries. The code works perfectly in development with expected inputs. It is trivially exploitable with a single quote character in production.
Example from a real audit. The AI generated:
const user = await db.query(
`SELECT * FROM users WHERE email = '${req.body.email}'`
);Instead of the parameterized version:
const user = await db.query(
'SELECT * FROM users WHERE email = $1',
[req.body.email]
);This is CWE-89: SQL Injection and it remains the most common critical finding in our AI code audits.
Detection: Run SAST scanning configured to flag string concatenation in database query contexts. Review every database call manually during code audit.
4. Hardcoded Secrets from Training Data
AI models are trained on public repositories. Those repositories contain committed secrets. When the model generates code, it reproduces patterns from training data, including placeholder credentials that look like real keys.
We have found AI-generated code containing strings like sk-proj-abc123... (OpenAI key format), AKIA... (AWS access key format) and JWT secrets set to your-secret-key or supersecret. These placeholders survive code review because the reviewing developer assumes they will be replaced by environment variables before deployment. Often they are not.
In one engagement we found a production Stripe secret key that the AI had generated as a "placeholder" and the developer had never replaced. The key was live and had access to payment processing.
Detection: Run entropy-based secrets scanning (tools like trufflehog or gitleaks) on every commit. Check git history for secrets that were committed and later removed.
5. Insecure Deserialization
AI assistants generate deserialization code without safety controls. Python's pickle.loads() on untrusted input, Java's ObjectInputStream.readObject() without type filtering and PHP's unserialize() on user-controlled data all appear regularly in AI-generated code.
These patterns enable remote code execution. An attacker sends a crafted serialized object to an endpoint that deserializes user input. The object executes arbitrary code during deserialization. This is CWE-502: Deserialization of Untrusted Data and it is consistently rated as a critical severity finding.
In a Python Flask application we audited, the AI had generated a caching layer that used pickle.loads() on data stored in Redis. Any user who could inject data into the cache could achieve full remote code execution on the server.
Detection: Search for pickle.loads, yaml.load (without Loader=yaml.SafeLoader), ObjectInputStream and unserialize in your codebase. Replace with safe alternatives or add strict type allowlisting.
What to Do About It
AI code assistants are not going away. They make developers faster. But they also introduce vulnerability patterns that traditional code review does not catch because the reviewer did not write the code and may not understand the security implications of what the AI produced.
The fix is targeted auditing. Not "run a scanner." Not "add AI to check AI." Manual security review by someone who knows what AI gets wrong and where to look. That is what we do at Sherlock Forensics.
Start with our free security checklist or scope an AI code audit starting at $1,500.