Why most AI support pilots stall

As soon as you open up your promised dashboard at the end of the first month of your AI pilot, you typically see the numbers that the pilot would have promised you. You see that the AI is handling a good share of the tickets, the queue is looking shorter, and the team is generally happy with how things are going. You know, the usual stuff that is promised.

However, when you check back in a few months’ time, you start to notice that the numbers haven’t really moved, that the team has gone back to handling most of the tickets the way they did before, and nobody is even checking the dashboard anymore.

Unfortunately, this is how most AI support pilots end, with everyone losing interest in it. So, it’s time to not fall for the same pattern. In this post, we’re going to share the three main reasons why we see this happen (with other people, not with us), and what successful pilots and implementation tend to look like.

What a stalled pilot actually looks like

In a study by Gartner, it expects that over 40% of the so-called agentic AI projects will be cancelled by the end of 2027. Now, that’s a really scary statistic to hear, when most of the world is still expecting the AI to head it in a different direction.

Klarna is a good example of both sides of this. In February 2024, the company announced that its AI assistant had handled 2.3 million conversations in its first month, which was two-thirds of all their customer service chats. Resolution times had come down from eleven minutes to under two, and repeat inquiries had dropped by 25%. About fifteen months later, their CEO told Bloomberg that they had gone too far, that they had focused too much on cost, and that the quality had suffered because of it. Klarna then started hiring people again.

Why do pilots stall after a good start?

Here are the three reasons we keep coming across:

1. The pilot was set up to impress

Teams usually pick the more interesting queries for a pilot, because those are the ones that look good in a demo. The problem is that interesting queries are also quite rare, so the pilot doesn’t actually show you the volumetric side of things.

2. After-sales service is kind of pathetic

There is usually somebody in charge during the buying process, but there is rarely anybody in charge afterwards. The correct way to do it is to review the failures every week, adjust the confidence settings, and continuously update the knowledge. It is not, and should not be, plug and play. Continuous monitoring and improving is what makes the implementation stick.

3. The wrong metric

The deflection rate goes up, and then somebody from operations points out that handle time has not moved at all, and nobody can explain the difference because the two numbers were never looked at side by side. Zendesk's Global CX Trends 2026 report found that 85% of CX leaders believe customers will leave a brand over an unresolved issue, even on the first contact. The deflection number on its own does not tell you whether a ticket was actually resolved, or whether it was just closed.

Deflection rate and resolution are somewhat related, so it makes sense that the increase in the former should meaningfully impact the latter. However, there comes a time in your pilot when the deflection rate has consistently gone up, but your customers are still unhappy, even furious.

How to run a pilot that survives

Here is how we would recommend approaching it:

1. Pick your highest-volume request type, even if it is a boring one. Password resets and access requests are not exciting, but that is where most of the time is going.
2. Give the pilot an owner who actually has time in their week for it, and put the weekly review in their calendar before the pilot even starts.
3. Agree on two numbers up front. One for the volume being handled, and one for how long the person on the other end is waiting. Always look at them together.
4. Book the review meeting at the very beginning. If there is no decision point on the calendar, the pilot will most likely never reach one.

Want to know more? Book a demo of Auralis AI today.