Human override rate

Human override rate is the share of decisions an AI agent escalated or proposed where the human reviewer reversed the outcome or materially changed it. The denominator is decisions that reached a human, not all conversations. The numerator is the subset where the human's answer differed in substance from what the agent had reached or recommended.

The metric measures escalation quality rather than escalation volume. Escalation rate tells you how many decisions the agent handed off. Override rate tells you whether those handoffs were the right ones and whether the agent's proposed answers were any good. Beam has framed this distinction directly, arguing that override rate on escalated decisions is the number that matters and suggesting a well-tuned agent should sit below 15%.

What the metric captures

Every escalation carries an implicit prediction. The agent is saying that this case exceeds its competence or its authority, and in most modern deployments it also passes along a draft answer, a recommended action, or a summary of what it believes the situation to be. The human either accepts that framing or replaces it.

Acceptance and replacement mean very different things. When a human accepts the agent's proposal without material change, the escalation was about authority rather than capability, and the case is a candidate for automation once policy allows it. When the human materially changes the outcome, the agent was wrong and the escalation caught the error. Override rate is the ratio between those two populations, which makes it a measure of how much the human layer contributes beyond a rubber stamp.

Material change is the term that needs a local definition. Rewording an answer is not an override. Reversing a refund decision, changing an eligibility determination, or contradicting the agent's account of the situation is. Teams that skip this definition end up measuring editing habits rather than decision quality.

How it differs from escalation rate

Escalation rate and override rate answer different questions and can be read together as a two-by-two. High escalation with low override describes an agent that hands off constantly and is usually right when it does, which means the escalation threshold is too conservative and there is automation headroom being left on the table. Low escalation with high override describes the dangerous quadrant: the agent rarely asks for help and is frequently wrong when it does, which strongly suggests it is also wrong on cases it never escalated.

Low escalation with low override is the target state, and high escalation with high override indicates a scope problem rather than a tuning problem, since the agent is being asked to handle a category it does not understand. None of these readings are available from escalation rate alone, which is why volume metrics on their own push teams toward suppressing handoffs rather than improving them. The escalation matrix governing where cases go should be revisited whenever a quadrant shifts.

The two failure modes it exposes

Over-escalation. The agent escalates cases it could have handled. This shows up as a low override rate concentrated in specific intents, where humans consistently confirm what the agent already proposed. The cause is usually a confidence threshold set uniformly across intents rather than tuned per intent, or an escalation policy written defensively during launch and never revisited.

Bad proposals. The agent escalates appropriately but its recommended action is frequently wrong, producing a high override rate. This is the more serious finding, because the same reasoning that produced bad proposals on escalated cases is producing unreviewed answers on everything below the threshold. A high override rate is evidence about the unescalated population too, and usually warrants pulling a sample of contained conversations into review before touching the threshold.

Both failure modes are invisible to containment rate, which counts only whether a human was involved. Accelate has noted that containment is the easiest of these metrics to inflate, by narrowing what the agent treats as in scope, and narrowing scope moves override rate not at all.

Instrumenting overrides without burdening reviewers

The cheapest instrumentation captures the agent's proposed outcome as a structured field at the moment of escalation, then compares it to the final outcome recorded when the case closes. Refund approved versus denied, order cancelled versus modified, eligibility confirmed versus rejected: wherever the outcome is already a structured value in the ticketing system, override computes automatically with no reviewer input.

For free-text answers, automatic comparison is harder and usually not worth building precisely. A practical substitute is a single optional control in the agent console asking whether the AI's recommendation was followed, filled in for a sampled subset rather than every case. A few hundred judgments a month is enough to track the metric without turning every handoff into a data-entry task. Some teams supplement this with model-based grading over transcript pairs, which is cheap enough to run at full volume, though it should be calibrated against human judgments before anyone reports it.

Either approach depends on the handoff carrying enough context for the reviewer to make a real decision. When a handoff arrives without the agent's reasoning or the customer history, reviewers rebuild the case from scratch and their conclusions become independent of the agent's, which inflates apparent overrides for reasons that have nothing to do with agent quality.

Reading the number alongside escalation rate

Override rate is a ratio over a self-selected population, so it moves when the population moves. Tightening the escalation threshold sends more borderline cases to humans and typically lowers override rate, because the added cases are ones the agent was nearly right about. Loosening it raises the rate. A change that coincides with a threshold change is therefore not evidence of anything until the two are separated.

The practical discipline is to report the pair together over the same intent mix, and to treat a benchmark figure such as the 15% Beam suggests as a rough orientation rather than a target to optimize toward. A team that manages override rate directly will get there by escalating more, which is not an improvement. Read alongside agent quality scoring and human-in-the-loop review coverage, the pair describes something no single metric does: whether the division of labor between the agent and the people behind it is drawn in the right place.