Red-Teaming Your AI Feature Before Launch
Run a focused adversarial test before shipping: attack categories, a test plan, automated probes, and fixing what you find.
Every lesson in this module added a defense: risk assessment, injection-resistant design, moderation, PII handling, safe rendering, abuse limits. Each one was tested in isolation, by the person who built it, against the attacks they thought of. That's exactly the testing that misses the failures users find in the first week.
Red-teaming is a structured, authorized attempt to make your own feature fail, run before launch by people trying hard to break it. It doesn't need a security firm or a month. For a typical AI feature, a focused effort of a few days finds the problems that would otherwise become screenshots on social media. In this lesson you'll scope a pre-launch red team, map attack categories to your feature, write a one-page test plan, build an automated probe runner with real detectors, run a short session with humans, and turn findings into fixes and a go or no-go decision.
What a pre-launch red team is for
Your eval suite from Module 9 measures how well the feature does its job on inputs you expect. A red team looks for the inputs you didn't expect, from people who don't want the feature to work as designed. The two feed each other: every failure the red team finds becomes an eval case, so the fix stays fixed.
A pre-launch red team answers one question: is anything bad enough, and easy enough to trigger, that we shouldn't launch until it's fixed? That framing keeps it focused. You're not trying to prove the feature is safe, which you can't. You're trying to find the worst problems while they're still cheap to fix, and to launch knowing what remains.
Write down the ground rules before anyone types an attack:
- Target: the exact feature, prompt version, and model, deployed to a staging environment that mirrors production.
- Data: seeded fake tenants and customers only. Never point attacks at real customer data or production systems.
- Markers: harmless, detectable payloads. A canary string planted in places the output should never reveal, a made-up discount code, a fake competitor name. You're proving the feature can be steered, not causing harm.
- Handling: findings are vulnerabilities. Keep them in a restricted tracker until fixed.
- Time box: for example, two days of automated probing and one 90-minute human session, then a triage meeting.