What Makes Something an AI Product?

Every product team has been handed some version of the same request: "Can we add AI to this?" Sometimes the answer is a feature users love within a week. Often it's a chatbot in the corner that nobody opens, or a "smart" field that's wrong just often enough that people stop trusting it. The difference rarely comes down to the model. It comes down to whether the feature was a good fit for a language model in the first place.

This lesson gives you a working definition of what a language model adds to software, a clear list of jobs it should not do, and a five-question test you can run on any feature idea in a few minutes. By the end, you'll be able to look at a product and say which AI features are worth building, and why.

What a language model adds to software#

Traditional software is excellent at anything you can specify precisely: calculate tax, sort by date, check a password, send an email at 9am. It breaks down when the input is messy human language or the rule is "you know it when you see it." That's where a large language model (LLM, a model trained on vast amounts of text to predict and generate language) earns its place.

In product terms, LLMs add four capabilities that were impractical to build before:

Notice what's missing from that table: facts. A model isn't a database. It knows a lot about the world from training, but that knowledge has a cutoff date, contains errors, and knows nothing about your customers. When an AI feature needs facts, your code supplies them (from your database, your documents, or a search) and the model works with what it's given. You'll build that pattern properly in Module 5.

Where models don't belong#

The fastest way to build a bad AI feature is to use a model for a job deterministic code already does well. Keep these out of the model's hands:

• Arithmetic and counting. Models can do simple math, but they're unreliable at counting dozens of items or summing a column. Compute totals, counts, and dates in code.
• Exact lookups. "What's the status of ticket 4821?" is a database query. Asking a model risks a plausible, wrong answer.
• Rules you can write down. If a refund is allowed only within 30 days and under a set amount, that's an if statement. Using a model makes it slower, costlier, and occasionally wrong.
• Anything needing a guaranteed result. Permissions, payments, security checks, and legal or medical determinations can't depend on a probabilistic component without human review.
• Very tight latency budgets. A model call takes from a fraction of a second to many seconds. Keystroke-level interactions need something faster or a design that hides the wait.

The pattern that works is a division of labor: the model handles the language and judgment, and your code handles the facts, math, rules, and decisions. A triage feature, for example, has the model label a ticket's topic and urgency, while code applies the routing rules and the SLA math. You'll see this split in nearly every lesson.

A five-question test for AI feature ideas#

Here's a quick test to run on any idea before you write code. Each question is a yes or no; a strong feature gets a clear yes on all five, and any no tells you exactly what to fix or why to walk away.

1. Is the hard part language or judgment? If the core difficulty is reading messy input, writing text, or applying fuzzy criteria, a model helps. If the hard part is data access, business rules, or speed, it won't.

2. Can a wrong answer be caught or cheaply undone? Models will sometimes be wrong. A good feature puts a human in the loop (a draft the agent edits), makes errors visible (a summary sitting above the original thread), or makes them cheap (a tag someone can change with one click). If a mistake is silent and expensive, redesign the feature or don't build it.

3. Can you tell a good output from a bad one? If you and your team can't agree on what "good" looks like for ten real examples, you can't improve the feature or notice when it breaks. Features with checkable outputs (a label, extracted fields, a summary judged against its source) are far easier to ship than open-ended ones.

4. Is it worth the cost and wait, every time it runs? Every call costs money and time. A summary that saves an agent two minutes of reading is a bargain. A model call on every page view to rephrase a static heading is not. Estimate how often the feature runs and what it saves the user.

5. Does it beat the boring alternative? Compare against the non-AI version: a form with a dropdown, a keyword search, a template, a rule. Sometimes the boring version is better: faster, cheaper, and predictable. The AI version has to win on what users actually care about.

Running the test on Deskmate#

Let's apply the test to six feature ideas for Deskmate, the help-desk app we use as a running example.

Two lessons hide in that table.

First, the same capability can be a good or bad feature depending on the design around it. Drafted replies and auto-sent replies use the same model call. The difference is entirely in question 2: who catches the mistake.

Second, the failures are usually obvious once you ask the questions. The settings chatbot sounds modern in a planning meeting, but nobody can say what a good answer is, and a well-designed page beats it. Asking "does it beat the boring alternative?" out loud kills a lot of weak ideas cheaply.

Features, not just chatbots#

When people say "AI product," they often picture a chat window. Chat is one interface, and it's the right one when users have open-ended questions they can't predict. But most valuable AI features don't look like chat at all:

• Invisible features: tickets arrive already tagged and routed; nobody types a prompt.
• Inline features: a "Summarize" button above a thread, an "Improve" action in an editor, a search box that understands meaning.
• Background features: a nightly job that extracts fields from every new contract.
• Generated artifacts: a weekly digest, a drafted reply, a filled-in form for review.

These tend to be easier to use, easier to evaluate, and cheaper to run than an open chat box, because the task is narrow and the input is known. Beyond the Chat Box in Module 8 covers how to choose an interface.

It also helps to place products on a spectrum. At one end are AI-assisted products: a conventional app with AI features that make existing workflows faster, like Deskmate with summaries and drafts. At the other end are AI-native products, where the model's output is the core value, such as a product-description generator for online stores. Most of what you'll build sits toward the assisted end, and that's where most of the reliable value is today.

You'll also hear the word agent: a model that runs in a loop, choosing tools and actions on its own until a goal is met. Some features need that autonomy, and Module 4 shows where a multi-step tool flow crosses into agent territory. Most product features don't; a single well-scoped call, or a short fixed chain of calls, is easier to test and cheaper to run. The Agent Builder Course explains when an agent beats a workflow if you want the full picture.

What makes it a product, not a wrapper#

If anyone can call the same model API, what stops your feature from being a thin "wrapper" a competitor copies in a weekend? The model is the part you share with everyone. What you build around it is yours:

• Your data and context: the tickets, documents, and history your feature can draw on.
• Your workflow: the feature sits exactly where users already work, at the moment they need it.
• Your quality system: evals, feedback loops, and the accumulated knowledge of how the feature fails.
• Your UX: interfaces that make the model's output easy to trust, check, and correct.

These are also the things the rest of this course teaches. Positioning: Beyond "Just a Wrapper" in Module 12 picks the topic up again from the business side.

ExerciseScore your own feature ideas

1. Take the three feature ideas you listed in the welcome lesson (or write three now for a product you know).
2. Add two more: one you suspect is a bad idea, and one that's currently done by hand by someone on your team.
3. Run all five through the five-question test. Use a table like the Deskmate one above.
4. For any idea that fails a question, write one sentence on how a redesign could fix it, or why it should be dropped.

Success looks like: at least one idea you'd confidently build, at least one you've decided to drop, and a clear reason for each.

Key takeaways

• Language models add four capabilities to software: understanding unstructured input, judgment on fuzzy categories, generating and transforming language, and flexible interfaces.
• Keep facts, math, exact lookups, rules you can write down, and guaranteed results in code. The model handles language and judgment; code handles everything else.
• Run every idea through five questions: Is the hard part language or judgment? Can errors be caught? Is quality checkable? Is it worth the cost? Does it beat the boring alternative?
• The design around the model often decides success. Drafts with review and auto-sent replies use the same call but carry very different risk.
• Most valuable AI features aren't chatbots, and what makes them defensible is your data, workflow, quality system, and UX.

Next: The AI Builder Stack in 2026 — a map of the layers every AI app is made of, and the sensible default at each one.