That elusive ROI in agentic AI
The headlines have been full of the craziness of tokenmaxxing as employees compete to burn the highest number of AI tokens to top AI spending leader boards. Microsoft got so fed up with the high cost of developer's use of coding agents, it cut off access to Claude Code. Meanwhile Uber reported burning through its AI budget in just four months. All of this spending surely must be generating a major return on investment. I mean how could it not.
But a recent report by Bain and Company tells a much different story. It turns out that it's hard to build autonomous agents, and even when you happen to succeed, it's still challenging to justify the hard cost of torching them tokens.
How bad was it? According to Bain, nearly 40% of the companies in their survey, who were measuring return (hard to believe anyone wasn't), reported ROI of between 0 and 10% after predicting (hoping, praying?) for between 11 and 20% return. Now you would think with abysmal returns like that, management might begin to question the wisdom of following the crowd down the AI rabbit hole, and yet…Bain found that 90% of respondents planned to increase their budgets. Lack of success be damned.
That's right, we may be losing money, or at least not coming close to our targets, but by golly we are going to double down on our losses like a feverish gambler at the Blackjack table unable to step away in the face of all logic and evidence to the contrary.
And it's not just Bain with these findings. Other firms like Deloitte and PwC have found similar results, so much so that you have to wonder why companies insist on throwing more money at the problem instead of stepping back and maybe figuring out a better way forward. There are best practices, but as the PwC report found, only a small percentage of companies are actually implementing them.
It's the data, stupid
When I started exploring the current generation of AI about three years ago, I remember having a conversation with a consultant, who told me he would go into companies to have an AI conversation, and it would very quickly become a data-readiness discussion.
Believe it or not, that's still the case. Bain found that 41% of respondents cited data access and integration as the biggest single factor holding back their AI progress. So three years later, it's still a data problem, although Bain is careful to point out that the data problem should not be a paralysis problem. You still have to try.
Another big issue that I've heard about continually is using AI to automate a process without looking at it in a new light. If you have the opportunity to remake an inefficient workflow with agents, don't just put an agentic wrapper around the old one. As the report states, "The question to ask before any AI program is approved is not 'Where can we apply AI?' but 'If we were designing this process from scratch today, what would it look like?'" That kind of approach from leadership is more likely to get rank and file employee buy-in because the folks doing the work know those inefficiencies better than anyone.
And even worse, perhaps, is that the finance people are not pushing back about the numbers. Bain recommends looking at returns from past automation efforts and using that as a benchmark. If anything, this generation of AI takes more effort, making it harder to project accurately. "Approving the next program on the assumption that [projected returns are real] without verifying it is the most avoidable financial risk in most companies' AI portfolios right now," the Bain report found.
That's what PwC found as well. Leading companies systematically track business impact and hold senior leaders directly accountable for AI outcomes, sensible business practices it seems that far too few companies actually follow.
It's ultimately about sticking to the basics. As I've written many times, enterprise fundamentals don't go out the window because we have new technology. Instead, they matter more than ever, whether it's prepping the data, reworking a process or trying to understand the underlying financial benefits (or lack thereof). It all comes back to fundamentals, and if you don't follow them you're very likely throwing good money after bad, and shouldn't expect to get a lot out of your agentic efforts.
~Ron
