The rollback rate is the wrong metric
Organizations are optimizing for failure prevention. The data says they should be optimizing for detection speed.
Guest edited by Gunnar Grosch, Developer Advocate at Sinch
The long read
By now, many people in this industry have come across the Air Canada chatbot story, or the viral clips from McDonald’s AI-powered drive-through.
When a customer-facing AI agent fails publicly, the story follows a familiar arc. Something goes wrong, customers notice, someone posts… and eventually the company rolls back the agent and announces it “acted immediately.”
What they actually mean is that they acted immediately after finding out. But the finding out, in most cases, took months. That gap, the one between an agent failing and your team noticing, is where the real risk lies.
Sinch research shows 74% of enterprises have rolled back a live AI customer agent, rising to 81% among those with the most mature guardrails. And the instinctive reading there is that governance investment isn’t working: Companies built safety controls, went live, and still had to pull the plug.
But is an organization at zero rollbacks, one that has never pulled a live agent, really running perfectly clean deployments? Probably not. What’s more likely is that no one has the visibility to see what’s happening.
The reason this gap between failure and detection exists is architectural
Standard infrastructure monitoring (latency, error rates, uptime…) was built to tell you whether a system is running, but it has nothing to say about whether the answers are right.
With AI agents, those are two very different things. An agent can return a 200 OK while telling a customer they’re eligible for a policy that no longer exists. The infrastructure records a success, but the customer hears something the company never intended to say.
Air Canada found this out in court following a lawsuit. McDonald’s rolled back their AI ordering technology after a wave of viral videos. By then, the reputational damage had already hit.
The industry treats these public incidents as AI model failures. In reality, they are observability failures. Sinch research puts a number on it: 17% of rollbacks came down to lack of audit trails, teams that couldn’t see what the agent had done.
Closing that gap means adding a second layer of observability, and this is where most teams get it wrong.
LLM observability isn’t an extension of the monitoring stack you already have. It requires different tooling entirely: Every tool invocation captured with full inputs and outputs, not summaries. Summaries don’t tell you what the agent decided or why. The payloads do.
That’s what lets you reconstruct what the agent was actually doing when something went wrong. But to actively catch those failures, you must run evaluations on your live traffic daily, rather than finding out from a customer complaint weeks later.
Most importantly, it takes a shift in how you think about these deployments, the same shift mature engineering teams already make with distributed systems. That means the question to ask is not: Will this fail? But: How fast will we know when it does?
The organizations rolling back at 81% aren’t the ones in trouble. They’re the ones who can see well enough to act. A rollback means detection happened, a decision was made, the failure was contained. That’s the system working.
The organizations at zero rollbacks are probably experiencing the same failure rate. They just haven’t found out yet.
Gunnar Grosch is a Developer Advocate at Sinch. Connect with him on LinkedIn.
Gunnar’s take
“An AI agent will happily return a 200 OK while actively hallucinating a policy that costs you thousands. We’re building massive incident response frameworks for systems that are architecturally blind. If you aren’t capturing the full payload of every tool call and evaluating live traffic daily, your observability strategy is just waiting to find out on social media.”
From the field
“Most agent teams still measure quality at deploy time and go blind afterward. What I have found on my teams is that detection requires treating agent decisions as first-class events. Every tool call and response lands in an append-only stream, so drift becomes a queryable anomaly instead of a support ticket. Sampled evaluations on live traffic close the loop, and time-to-detect a wrong answer becomes a metric you can put an SLO on.”
Go deeper
This issue’s question
If your AI agent failed silently in production this week, how long would it take your team to find out?
Email us and share your story with us.
In Production is a monthly editorial newsletter on AI in customer communications, written and published by Sinch. Read past issues on Sinch.com and catch the newsletter on LinkedIn.
Author: