Safety, Privacy & Responsible AI
Shipping AI features you can stand behind
Protect users, data, and your business: risk assessment, prompt injection in apps, moderation, PII and provider data policies, securing model output, abuse and denial-of-wallet, AI regulation, copyright and provenance, and pre-launch red-teaming.
- Lessons
- 9
- Reading time
- 92 min
- Level
- Advanced
- Access
- Full course
Lessons in this module
- 11.111 min
Risk Assessment for AI Features
A lightweight method to find what could go wrong with a feature — who's harmed, how badly, how likely — and pick proportionate mitigations.
- 11.211 min
Prompt Injection in Apps: RAG, Uploads, and User Content
How injected instructions reach your app through documents, emails, and web pages — and the design patterns that limit the damage.
- 11.310 min
Input and Output Moderation
Screen what goes in and what comes out: moderation APIs, classifier prompts, policy definitions, and handling false positives without frustrating users.
- 11.410 min
PII, Data Retention, and Provider Policies
Know where user data flows: provider retention and training policies, zero-data-retention options, PII redaction, regional processing, and user consent.
- 11.59 min
Securing Model Output: XSS, Links, and Markdown
Treat model output as untrusted input to your UI: sanitizing HTML and markdown, link and image exfiltration, and safe rendering of generated code.
- 11.612 min
Abuse Prevention and Denial-of-Wallet
Stop free-tier abuse, scraping, jailbreak farming, and attacks designed to run up your bill — with limits, signals, and cheap first-line filters.
- 11.710 min
AI Regulation for Builders
What the EU AI Act and other rules mean in practice for small teams: risk tiers, transparency duties, and the documentation worth keeping anyway.
- 11.810 min
Copyright, Licensing, and Content Provenance
Terms of use for model output, open-weight licenses, training-data questions, and content credentials for generated media — what builders need to know.
- 11.99 min
Red-Teaming Your AI Feature Before Launch
Run a focused adversarial test before shipping: attack categories, a test plan, automated probes, and fixing what you find.
About this module
AI features fail differently from ordinary software. The code can be correct while the feature produces a plausible but wrong answer, follows instructions hidden in a customer's document, leaks data through a rendered image link, or runs up a bill nobody planned for. Safety, privacy, and security for AI products is the practice of finding those failures before users do and designing features so that the ones you miss can't do much harm.
This matters for every team that puts a model in front of real users, not only for companies in regulated industries. Customers' security teams ask where their data goes and whether it trains anyone's model. Attackers have learned that any text a model reads can steer it. Regulators increasingly expect documentation and transparency for certain uses. And a single screenshot of an AI feature behaving badly can undo months of trust.
This module works through the risks in the order a product team meets them. It starts with a lightweight risk assessment that scores what could go wrong with a feature and picks proportionate mitigations. It then covers prompt injection through RAG, uploads, and user content; input and output moderation; PII, data retention, zero data retention, and provider policies; and treating model output as untrusted input to your UI, including XSS and markdown exfiltration.
The second half covers abuse prevention and denial-of-wallet attacks, what the EU AI Act and other rules mean in practice for small teams, copyright, licensing, and content provenance for generated output, and how to red-team a feature before launch. Every lesson is practical, with TypeScript code, checklists, and exercises you can apply to a real product, using a help-desk app as the running example.
Questions about Safety, Privacy & Responsible AI
What is prompt injection in an AI app?
Prompt injection is when text from an untrusted source, such as a support ticket, uploaded PDF, retrieved document, or web page, is treated by the model as instructions. It can flip classifications, insert phishing links into drafted replies, or leak data from the prompt. There is no complete fix, so defenses focus on narrowing outputs, limiting data and capabilities, and keeping humans between untrusted input and consequences.
Do AI providers train on data sent through their API?
At the time of writing, the major providers' paid API terms say customer API data is not used for training by default, but free tiers and consumer apps can differ. Google's Gemini API, for example, may use unpaid-tier content to improve its products. Always check the current data terms for the exact account and tier you use.
What is zero data retention for LLM APIs?
Zero data retention (ZDR) is an arrangement where the provider does not store your prompts and outputs after processing, apart from narrow exceptions such as legal obligations. It usually requires approval or an enterprise agreement and excludes features that store data by design, such as file storage. It only covers the provider; your own logs, caches, and vector store still need retention rules.
How do I stop XSS from LLM-generated markdown?
Render markdown into React elements with a library that ignores raw HTML instead of converting it to an HTML string and injecting it. If you need real HTML, sanitize it with a strict allowlist such as DOMPurify. Also allowlist image hosts and show link destinations, because markdown images and links can leak data even without script execution.
Should I use a moderation API or an LLM to moderate content?
Dedicated moderation APIs are fast and cheap for standard harm categories like harassment, violence, and self-harm. An LLM classifier given your written policy handles product-specific rules, such as not giving legal advice, that fixed taxonomies can't. Many teams use both, letting the model label content and their code decide the action.