What does 'in production' mean for an AI system?
Real inputs flow automatically, an eval set of real cases exists and the release passed it, guardrails cover high-impact failures, uncertain cases route to people, decisions are logged, a dashboard tracks volume, quality, latency and cost, and a named owner can operate it from a runbook.
Write the definition down before building. A reasonable bar is that real inputs flow automatically from source systems, the current version passes an eval set of real cases, guardrails cover the high-impact failure modes, uncertain cases go to a human queue, every decision is logged, a dashboard shows volume, quality, latency and cost, and a named owner reviews the metrics on a schedule with a runbook for failures.