Building AI Features That Survive Production: Start With Evals, Not Prompts

The demo is the easy part

Almost any modern model can produce one impressive answer in a notebook. Production is a different sport: traffic arrives in bursts, users paste in malformed input, providers throttle you, and a silent model update changes behavior you depended on. None of that is a prompting problem. It is a systems problem, and it deserves the same engineering discipline we apply to any other backend service.

The shift in mindset is simple. Treat the model as an unreliable but useful component behind an interface, the way you would treat a third-party API that occasionally times out. Your code owns validation, retries, fallbacks, and the definition of a correct answer.

“If you cannot measure an AI feature, you are not maintaining it. You are just hoping near it.”

Evals are unit tests for behavior

Start with twenty to fifty real examples: actual user requests, the answer a human would accept, and the answers you would reject. Score every prompt, model, or retrieval change against that set before it ships. Keep the set small enough to run on every pull request and honest enough that a green run means something.

Add the boring production counters next: latency, error rate, cost per successful answer, and how often users accept the output without editing it. When a change improves the eval score but doubles cost per success, you want that argument to happen in a dashboard, not in a finance review.

interface GoldenCase {
  id: string;
  input: string;
  mustContain: readonly string[];
  mustNotContain: readonly string[];
}

function scoreCase(testCase: GoldenCase, output: string): boolean {
  const text = output.toLowerCase();
  return (
    testCase.mustContain.every((term) => text.includes(term)) &&
    testCase.mustNotContain.every((term) => !text.includes(term))
  );
}

Ship it like a service, because it is one

Version your prompts the way you version schemas, and log which version produced which answer. Roll changes out to a small slice of traffic first. Keep a boring fallback ready: a cached answer, a simpler model, or a graceful message to the user. The goal is that a bad AI change becomes a minor incident with a rollback button, not a mystery that takes a week to notice.

None of this makes the feature less magical for users. It just means the magic has tests, budgets, and an off switch, which is what separates a product from a demo.