Building AI Features Users Actually Want

There's a pattern I see constantly in the projects that land on my desk: "We want to add AI to our product."

When I ask what they mean, I get one of a few answers: a chatbot, an AI-generated summary, or "something like what ChatGPT does but for our use case."

None of these are wrong, but they're solutions looking for problems. And that's exactly backwards.

The three questions

Before you build any AI feature, answer these:

1. What's the job to be done? Not "what could AI do for us?" — what is the specific task a user is trying to accomplish, and where are they getting stuck? AI is a capability. It's only valuable when it solves a real friction point.

2. What does failure look like? AI systems fail differently than traditional software. They fail probabilistically. They hallucinate. They degrade at the edges of their training distribution. If a wrong answer in your context causes a user to lose money, trust a bad medical recommendation, or make an irreversible decision — the bar for reliability is much higher than you think.

3. What's the alternative? If the alternative to your AI feature is "the user does it manually," you need to ask how often they do it, how long it takes, and how much they'd pay to not do it. If the alternative is "nothing" — the task just didn't exist before — you're probably building something users won't use.

Where AI actually works

The sweet spots I've found in real products:

Tedious classification at scale. If a human has to read something and put it in a bucket (support ticket → category, document → type, image → label), AI can do this at volume with acceptable error rates. The key: make the error cost low and the error visible.

First draft generation. AI is excellent at producing a starting point that a human then refines. Email drafts, code skeletons, report summaries. The human stays in the loop, but the blank page problem disappears.

Search and retrieval across unstructured data. RAG (Retrieval-Augmented Generation) pipelines let users ask questions over a corpus of documents in natural language. This is genuinely hard to replicate without AI.

Anomaly flagging. "Here's a thing that looks unusual — you should look at it." Low stakes if the model is occasionally wrong, high value when it catches something a human would have missed.

A worked example

A legal-tech client came to me with a version of the classification problem above: a team was spending 40+ hours a week manually pulling clauses and metadata out of contracts that varied wildly in format and language. A rules-based system had already failed once. The fix wasn't a single clever prompt — it was breaking the job into stages a model could actually do reliably: classify the document type first, then detect its structure, then extract only within that structure. Each stage has a narrower job and a lower error rate than trying to do everything in one pass. That's "tedious classification at scale" from above, applied for real: the error cost stays low because no single prompt is ever asked to do five jobs at once.

The chatbot trap

Chatbots are often the wrong answer. Not because they can't work — they can — but because they're high effort, high maintenance, and they shift the entire burden of formulating a good query onto the user.

A well-designed UI with smart defaults and good filtering will beat a chatbot interface for most use cases, because the product has already done the thinking for the user.

Chatbots make sense when the problem space is genuinely open-ended, when the user's needs are highly variable, and when you've already exhausted structured UI options.

The reliability gap

Here's the thing nobody tells you when you start building AI features: users' tolerance for AI errors is much lower than engineers expect.

If a search engine returns an irrelevant result, users try a different query. If an AI assistant gives them wrong information confidently, they feel betrayed.

Design for the failure case explicitly. Show confidence levels. Add "verify this" prompts. Make it easy to report problems. Keep a human in the loop for high-stakes decisions.

I apply the same discipline to my own AI projects. Cognitive Edge, a content pipeline I built that gathers and summarizes psychology, business, and tech articles daily, keeps and credits the original source on every single item it publishes — not as an afterthought, but as a hard rule in the pipeline. If the model mis-summarizes something, the source is right there for a reader to check. That's the "verify this" prompt from above, built into the product instead of bolted on as a disclaimer.

// Don't just display the AI output
const Summary = ({ aiOutput, confidence }: Props) => (
  <div>
    <p>{aiOutput}</p>
    {confidence < 0.8 && (
      <p className="text-amber-600 text-sm">
        ⚠ This summary may not be accurate. Please review the source document.
      </p>
    )}
  </div>
);

The meta-lesson

The teams that ship great AI features treat it like any other product problem: they start with the user, they define success concretely, and they design for the full range of outcomes — including failure.

The teams that ship AI features nobody uses start with the technology and work backwards.

The technology isn't the hard part. The product thinking is.

Frequently asked questions

How do I know if an AI feature is worth building? Answer the three questions above first — the job to be done, what failure looks like, and what the real alternative is. If you can't answer the first one concretely, the feature isn't ready to build yet, no matter how impressive the demo looks.

What's the biggest AI product mistake you see? Defaulting to a chatbot as the interface. It shifts the entire burden of asking a good question onto the user, and it's usually a worse experience than a well-designed form or filter that already did the thinking for them.