1. Trust & Hallucinations

AI sounds confident even when it is wrong. It doesn’t show uncertainty — and may invent APIs, libraries, or features that don’t exist.

# AI generated — 100% confident:
def connect_database():
    import mysql.connector
    # ⚠️ Deprecated library!
    conn = mysql.connector.connect(...)

Always verify:

• Library names, method signatures, and versions against official docs
• Security-critical code (encryption, auth, authorization)
• Edge cases, error handling, and return values

Treat AI output as a draft, not as truth.

2. The Context Problem

AI generates code that doesn’t match your project’s architecture.

# Your project: Repository Pattern
class UserRepository:
    def __init__(self, db_session):
        self.db = db_session

# AI suggests: Active Record ❌
class Product:
    def save(self):
        db.session.add(self)

Fix it once — in your rules files:

• .cursorrules / CLAUDE.md / copilot-instructions.md

Encode: architecture patterns, naming, tech stack, security requirements, testing expectations.

Fix the instruction layer, not just the output.

3. The Fundamentals Trap

Developers use AI as a hiding place instead of a tool.

❌ “The AI did this, I don’t know why it works”

❌ Copy-paste without reading

❌ Debugging by “asking AI” instead of investigating

❌ Accepting code you could not explain in a code review

4. Security & Privacy Risks

AI regularly generates code with security issues.

# ❌ SQL Injection
query = f"SELECT * FROM users WHERE name = '{username}'"

# ❌ Hardcoded secret
API_KEY = "sk-proj-abc123..."

# ❌ PII in logs (GDPR violation)
logger.info(f"User {email} performed {action}")

Privacy pitfalls:

• Pasting secrets or customer data into prompts
• Using model output directly without validation
• Allowing AI tools overly broad permissions

Security is NOT delegable.
If code touches identity, payments, or personal data — treat AI output as untrusted until proven otherwise.

Security Checklist

Before merging AI-assisted code:

• SAST tools in CI (Bandit, Semgrep, Snyk)
• Dependency scanning (Safety, Dependabot)
• Secret scanning before merging (GitLeaks, TruffleHog)
• Manual security review for critical flows
• Input validation and output encoding at all trust boundaries
• GDPR impact assessment when personal data is involved
• At least one human has reviewed and understood the change

5. Missing Tests & False Confidence

AI makes code look finished before it’s exercised. It generates the happy path and forgets what breaks production: empty arrays, null values, race conditions, malformed input.

Better workflow:

• Write the test first, let the model implement against it
• Run full suite after every change
• Add a test for every bug AI introduces
• Test the requirement, not the implementation

6. Cost Explosions

How to keep costs down:

• Minimize context — don’t send entire repos
• Improve prompts instead of regenerating repeatedly
• Use cheaper models for simple tasks
• Long chats are expensive — they carry all old context

7. Vendor Lock-in

Some tools tie you to specific platforms:

Before adopting a tool, ask:

• Can I export the code and run it elsewhere?
• Does the workflow depend on a vendor-specific API?
• Can I replace the model without redesigning the app?

8. The Maintenance Problem

AI can generate code that works today but is expensive to maintain tomorrow.

Warning signs:

• Huge generated files with no clear boundaries
• Duplicated logic from repeated “just add one more thing” prompts
• Code that ignores the project’s naming and layering conventions
• A feature that works, but only if you never need to change it

Better habits:

• Ask for small, modular changes
• Refactor while context is fresh
• Keep functions short and responsibilities clear
• Prefer boring code over magical code

The Practical Checklist

Before accepting AI-generated code, ask:

• Does this match the project’s architecture and conventions?
• Can I explain what the code does and why it is correct?
• Did I verify the API, library, or version against the docs?
• Are the security, privacy, and compliance implications acceptable?
• Do I have tests for the important behaviour and edge cases?
• Would I still merge this if there was no AI author to blame?