1. The Trust Problem

AI sounds confident even when it is wrong. It does not show uncertainty or tell you when it is guessing.

# AI generated — 100% confident:
def connect_database():
    import mysql.connector
    # ⚠️ Deprecated library!
    conn = mysql.connector.connect(...)

Always verify:

• New or obscure libraries and APIs
• Security-critical code
• Performance-critical paths
• Edge cases and error handling

Treat AI output as a draft, not as truth.

2. The Context Problem

AI generates code that doesn’t match your project’s architecture.

# Your project: Repository Pattern
class UserRepository:
    def __init__(self, db_session):
        self.db = db_session

# AI suggests: Active Record ❌
class Product:
    def save(self):
        db.session.add(self)

Fix it once — in your rules files:

• .cursorrules
• AGENTS.md
• copilot-instructions.md

Encode: architecture patterns, naming, tech stack, security requirements, testing expectations.

Fix the instruction layer, not just the output.

3. The Hallucination Problem

The model invents APIs, options, and flags that don’t exist.

Typical symptoms:

• The API call does not exist
• Package name is valid, but the feature is from a different package
• Flags removed in a newer version
• Code depending on behaviour that was never guaranteed

What to do:

• Verify against official documentation
• Ask the model to explain each step in plain language
• Be extra cautious with obscure APIs or brand-new releases
• Prefer code you can check with tests and type-checkers

4. The Fundamentals Trap

Developers use AI as a hiding place instead of a tool.

❌ “The AI did this, I don’t know why it works”

❌ Copy-paste without reading

❌ Debugging by “asking AI” instead of investigating

❌ Accepting code you could not explain in a code review

5. Security & Compliance

AI generates code with well-known vulnerabilities.

# SQL Injection
query = f"SELECT * FROM users
  WHERE name = '{username}'"

# Hardcoded secret
API_KEY = "sk-proj-abc123..."

# PII in logs (GDPR!)
logger.info(
  f"User {email} performed {action}")

Be especially careful with:

• Authentication / Authorization
• Input validation
• Data encryption & secrets
• GDPR / DSGVO compliance
• Output encoding

Security is not delegable.
AI makes these mistakes regularly.

Security Checklist

Before merging AI-assisted code:

• SAST tools in CI (Bandit, Semgrep, Snyk)
• Dependency scanning (Safety, Dependabot)
• Secret scanning before merging (GitLeaks, TruffleHog)
• Manual security review for critical flows
• Input validation and output encoding at all trust boundaries
• GDPR impact assessment when personal data is involved
• At least one human has reviewed and understood the change

If the code touches identity, payments, or personal data — treat AI output as untrusted until proven otherwise.

6 & 7. Tests & Cost

Missing Tests & False Confidence

AI makes code look finished before it’s exercised.

• Write the test first
• Run full suite after every change
• Add a test for every bug AI introduces
• Test the requirement, not the implementation

Cost Explosions

• Minimize context (relevant files only)
• Use caching where available
• Match model to task

8 & 9. Lock-in & Privacy

Vendor Lock-in

• Keep code portable
• Use abstraction layers
• Always plan your exit strategy

Privacy & Prompt Injection

• Don’t paste secrets, PII, or internal code into prompts
• Validate and sanitize every model output reaching users
• Restrict tool permissions
• Assume prompt injection is possible whenever user-controlled content is involved

10. The Maintenance Problem

AI can generate code that works today but is expensive to maintain tomorrow.

Warning signs:

• Huge generated files with no clear boundaries
• Duplicated logic from repeated “just add one more thing” prompts
• Code that ignores the project’s naming and layering conventions
• A feature that works, but only if you never need to change it

Better habits:

• Ask for small, modular changes
• Refactor while context is fresh
• Keep functions short and responsibilities clear
• Prefer boring code over magical code

The Practical Checklist

Before accepting AI-generated code, ask:

• Does this match the project’s architecture and conventions?
• Can I explain what the code does and why it is correct?
• Did I verify the API, library, or version against the docs?
• Are the security, privacy, and compliance implications acceptable?
• Do I have tests for the important behaviour and edge cases?
• Did I keep the prompt and context small enough to stay focused?
• Would I still merge this if there was no AI author to blame?