Two failures, two disciplines

September 10, 2026

When an AI agent breaks in production, everyone’s instinct is to ask how to fix it. But that’s the wrong first question to ask. The right one is: Which kind of break is this? Because there are two types of failures, and they’re ultimately nothing alike.

The single biggest cause of a rollback isn’t the model at all, it’s PII exposure, at 31%. A customer’s account details surfacing when they shouldn’t is an infrastructure failure. Or their history vanishing the moment they move from web chat to email, forcing them to start over.

Hallucination and off-brand answers, the failure everyone braces for, come in behind at 22%. That’s a model failure. The agent misreads what the customer wants, or answers confidently and wrong under your brand’s name.

These failures have different causes that require different fixes. Tune the prompt on what was really a data-scope problem, and the leak will still be there.

But telling them apart is harder than it sounds. Between the moment an agent takes something in and the moment it acts, there’s a gap you can’t see into. Just like Gunnar Grosch explained in last month’s issue of In Production, you can monitor what goes in and what comes out, but not the reasoning in between. So, when it breaks, your logs hand you the result, but they don’t tell you the cause.

Here’s why the split is important. The two failures live on opposite sides of one line: what you control, and what you don’t. A model is probabilistic. That means you can steer it and shrink its error rate, but you’ll never get it to zero, because its output is never fully nailed down.

Everything outside of the model is the opposite. What data goes in, what the agent is allowed to do, which gates are open. Somebody decides all of that… or forgets to. That is not probabilistic. It is a design choice.

You can ask an agent to mask a credit card number in its reply, and it usually will. Or you can make sure the number never reaches the model. If you mask it upstream, in the data you choose to pass along, it can’t leak… because it was never there.

One is a request. The other is a fact. Infrastructure failures are the ones we can design out at the platform level, so the model is never in a position to get them wrong.

The model failures you can’t eliminate, so you need to build to contain them. First, give the agent one narrow job and a short list of things it’s allowed to do. The shorter that list, the fewer ways a probabilistic decision turns into a real consequence, and the faster you can spot which move went wrong.

Next, put a guardrail check before each action, not only at the end. Monitor the two points you actually can see, the input and the output, and build a database of “low confidence” outcomes for a human expert to check and answer.

Then, close the loop. Feed the expert’s answers back to the model for further training, because a mistake the system doesn’t learn from comes back in the same shape next week.

But there’s a cost that we need to consider: If we constrain too far (withhold too much data or scope it too narrowly), the agent starves because context is what makes it useful. So, where that line sits and where the right balance lies is a judgment you can’t hand off.

Two failures modes, two disciplines. Design out what you own; box in what you don’t. The teams that can do neither are usually the ones that can’t tell the two failure modes apart to begin with. Sinch research shows that for 16% of organizations, a failure leaves no record of what the agent decided or touched, so every break becomes a full rollback and a rebuild of the same fault. A recent Sinch poll on LinkedIn showed similar results.

None of it is magic. Every time we build an agent, someone defines the data, the actions, the gates. The only question is whether it’s you, on purpose… or the model, by default.

In Production is a monthly editorial newsletter on AI in customer communications, written and published by Sinch. Read past issues on Sinch.com and catch the newsletter on LinkedIn.