AI customer support is a good tool and a bad headcount plan. The first dashboard always looks wonderful, because it counts the chats the bot touched. The second dashboard, three or six months later, counts the customers who came back.
Klarna is the case study every support leader has been handed this year, and it’s worth reading in order. In February 2024, Klarna said its AI assistant had handled 2.3 million conversations in its first month, about two thirds of its customer service chats, and was doing the work of 700 full-time agents. In May 2025 its chief executive told Bloomberg the company was hiring humans again. Both things are true, and the gap between them is the lesson.
What did Klarna actually claim in 2024?
Read the claim carefully, because it’s a claim about volume. Klarna reported 2.3 million conversations in a month. It also reported that repeat inquiries fell by 25 percent and that errands were resolved in under two minutes, compared with eleven before, according to coverage such as The Decoder. Those are real numbers and they point the right way.
Now do the arithmetic the headline invites. Divide 2.3 million conversations by 700 agents and you get about 3,286 conversations per agent per month. A month has about 21 working days, so that’s 3,286 ÷ 21 = roughly 156 conversations per agent per working day. That’s my back-of-envelope sum, not Klarna’s. It shows what “the work of 700 agents” means: a volume equivalence, not a count of customers who were fully helped, and not a count of jobs.
That’s the first uncomfortable point. A bot can take a conversation and still leave the customer where they started. Volume handled is an input. Whether the problem stayed fixed is the result.
What changed in 2025?
In May 2025, as Maginative reported from the Bloomberg interview, Klarna’s chief executive Sebastian Siemiatkowski said cost had become too dominant a factor in how the company judged its approach, and that the result was lower quality. Klarna began recruiting human customer service staff again so customers would always be able to reach a person. Other coverage notes the company still expects to use AI heavily, and the rehiring was described as an early pilot.
The wider data agrees with the direction. Gartner polled 163 customer service leaders in March 2025 and found that 95 percent planned to keep human agents. In June it predicted that half of the organisations planning big cuts to their customer service workforce because of AI would drop those plans by 2027, as The Register summarised. Companies hadn’t found the agent-less desk they were promised.
I thought the interesting question was how much of the queue AI could take. It’s how well the people behind it are set up to take over.
How do you measure cost without fooling yourself?
Start with what the first dashboard usually shows: cost per contact. Take total monthly cost and divide by the number of contacts. Say a desk spends $41,000 a month on its AI platform and its people together and receives 2,000 contacts. That’s $41,000 ÷ 2,000 = $20.50 per contact. It looks fine, and it hides everything that matters.
The better figure is cost per contact that was resolved and stayed resolved. Count only the contacts that ended in a fix where the customer did not come back about the same problem within seven days. If 1,250 of those 2,000 contacts qualify, the cost is $41,000 ÷ 1,250 = $32.80. The cost didn’t change. The denominator did, and it now only counts customers who got what they came for.
The same trap sits behind other averages. I wrote about it in Average Handle Time: The Most Misused Metric in the Contact Centre. A fast contact that has to be repeated is more expensive than a slow one that’s done once, and a bot is very good at being fast.
The next piece covers the full report, with the spreadsheet laid out step by step. For now, remember the rule: count what stayed fixed, not what was touched.
What should the human side of the desk look like now?
If the bot takes the simple, repeatable questions, the people take what’s left, which is harder, angrier and more ambiguous. That changes the job. I covered the routing side in Skill-Based Routing With AI Agents in Front, and the staffing consequences follow from it.
Four things I’d have in place before expanding any AI front line.
A hand-off that carries the story. When the bot passes a customer to a person, the person sees the whole conversation and the customer doesn’t repeat a word. If your tool can’t do that, fix it before you add volume.
A visible way out. A customer should be able to reach a human in one step, without typing the right magic phrase. Klarna’s own reversal was about making sure that option was always real.
Someone who owns the answers. Every topic the bot handles needs a named person who reads a sample of its replies each week and fixes the source article when it’s wrong. The knowledge base work I described this spring comes down to the same idea: the bot is only as good as the content someone is responsible for.
Staffing for the hard queue. The contacts that reach people are longer and heavier. Staff to the new handle time, not the old one, or the human queue quietly becomes the bottleneck.
How should you pilot it before you commit?
Run a small, honest test on one topic. Password resets and order status are good first choices, because the right answer is clear and a wrong one is cheap to fix.
Say you route 500 contacts on that topic to the bot over two weeks, and keep a comparison group of 500 with your agents. Afterwards, count the contacts in each group that were resolved and did not come back within seven days. Suppose the bot group scores 410 and the agent group scores 455. Divide each by 500: 410 ÷ 500 = 0.82, or 82 percent, against 455 ÷ 500 = 0.91, or 91 percent. The bot is 9 points behind on that topic.
Then price it. If a bot-handled contact costs $1 and an agent-handled contact costs $9, the cost per contact that stayed resolved is $1 ÷ 0.82 = about $1.22 for the bot and $9 ÷ 0.91 = about $9.89 for the agent. The bot is still far cheaper on this topic, even though it’s less accurate. That’s a good reason to use it here, and a reminder that the answer changes by topic. The same sums on a billing dispute may point the other way.
Those costs are invented to show the method. Use your own. What matters is that you compare like with like, and that the denominator is contacts that stayed fixed.
What does this mean for the people on the desk?
It changes the work before it changes the headcount. The agents who stay will take the contacts the bot couldn’t close, which are the longer, more emotional and more technical ones. Their day gets harder even if the queue gets shorter.
Tell them that. Then use their knowledge. The people who handle the hand-offs know exactly where the bot fails, and they can give you the best list of topics to fix or take away from it. Give someone the job of collecting that list weekly. It costs an hour and it’s worth more than most of the vendor’s reports.
A team that believes it’s being replaced stops reporting problems. A team that’s asked to shape the system keeps reporting them. That difference shows up in your numbers within a quarter.
What would I do this quarter?
Pick one topic where the bot already answers customers. Pull 50 conversations at random from last month and read them. For each one, mark whether the customer’s problem was actually solved and whether they came back within seven days. Divide the solved ones by 50. That number, not the volume, tells you what the bot is worth on that topic.
Then put the result next to the cost. If the figure is good, expand carefully, one topic at a time. If it isn’t, you’ve found the fix before your customers did. Either way you’ll be better placed than the team that’s counting chats, and you’ll have a number for the conversation about whether support is a cost centre that doesn’t fall apart on first questioning.
Klarna’s lesson isn’t that AI doesn’t work. It’s that cost is easy to count and quality is harder, and the easy number wins unless you decide otherwise.
Hutch Morzaria is a Director-level CX and Support Leadership professional with 19 years of experience building global support organizations across SaaS, Fintech, and enterprise technology. He has hired dozens of support and CX leaders across his career and holds ITIL Expert certification across V3 and V4.
