Every mature engineering organization has an incident process. Severity levels, an on call rotation, a bridge line, a blameless postmortem template. That machinery was built for a specific class of failure: something that worked yesterday stopped working today, and a change caused it. Roll back the change, restore service, write it up. It is a good process and it does not survive contact with a production AI system.
It fails quietly, not loudly
The first thing that breaks is detection. A deterministic service fails loudly. It returns a five hundred, the error rate spikes, the alert fires. An AI system usually fails quietly and confidently. The endpoint stays healthy, latency looks normal, and the responses are grammatically perfect and materially wrong. By the time someone notices, the system may have been producing bad output for days. This is why the trigger for an AI incident is rarely a monitoring alert and far more often a customer complaint or an internal user who happened to know the correct answer.
The change that broke it may not be yours
The second thing that breaks is root cause. Traditional incident analysis assumes a change caused the failure, so you walk the deployment history until you find it. With AI systems the change may not be yours. A provider updated a model version behind a stable alias. A retrieval index silently drifted as source documents were revised. An upstream schema shifted and the context assembled for the model quietly lost a field. None of these appear in your release log. Effective triage for AI incidents means asking not just what did we deploy, but what changed underneath us, and having enough version pinning and provenance capture in place to answer that.
There is no clean rollback for a behavior
The third thing that breaks is remediation. Rolling back a bad deploy is a known operation. Rolling back a model behavior is not, because the behavior is emergent from the model, the prompt, the retrieved context, and the input all at once. Teams that handle this well have prepared the levers in advance: a pinned previous model version they can route back to, a feature flag that drops the system into a deterministic or human reviewed path, and a documented decision on when degraded service is preferable to confident error. Those levers have to exist before the incident, because they cannot be built during one.
The blast radius outlives the outage
The last piece is scope. When a deterministic service fails, the blast radius is the outage window. When an AI system gives bad guidance, the blast radius is every decision made on that guidance, which may extend well past the fix. That turns remediation into a records question. Which outputs were affected, who acted on them, and what has to be corrected or disclosed.
Write the runbook before you need it
Treat your AI systems as a distinct incident class with their own severity definitions, their own runbooks, and their own postmortem questions. The organizations that write those documents before the first serious incident are the ones that stay in control during it.
