FastForward #68: That elusive ROI in agentic AI
Hi everyone. I'm writing from the west coast this week. The enterprise AI picture keeps getting more more complicated. Everyone's doing it, but not many companies are seeing results. I dig into why this week. Just a heads up that next week is a work week, so there will be no newsletter next Friday. I'm moderating a CEO fireside at Pax8 Beyond in Salt Lake City on Monday, then I'm off to Vegas for the Zscaler Zenith conference where I'll be doing a video interview with their EVP of the Agentic AI Security Startup division. Both of these are paid gigs. If you need a professional moderator for your event, drop me a line at ron@fastforward.blog. If you like my newsletter, please share this week’s edition with a friend, it really helps.💌 Sign up here.
ForwardThinking 🤔
That elusive ROI in agentic AI
The headlines have been full of the craziness of tokenmaxxing as employees compete to burn the highest number of AI tokens to top AI spending leader boards. Microsoft got so fed up with the high cost of developer's use of coding agents, it cut off access to Claude Code. Meanwhile Uber reported burning through its AI budget in just four months. All of this spending surely must be generating a major return on investment. I mean how could it not.
But a recent report by Bain and Company tells a much different story. It turns out that it's hard to build autonomous agents, and even when you happen to succeed, it's still challenging to justify the hard cost of torching them tokens.
How bad was it? According to Bain, nearly 40% of the companies in their survey, who were measuring return (hard to believe anyone wasn't), reported ROI of between 0 and 10% after predicting (hoping, praying?) for between 11 and 20% return. Now you would think with abysmal returns like that, management might begin to question the wisdom of following the crowd down the AI rabbit hole, and yet…Bain found that 90% of respondents planned to increase their budgets. Lack of success be damned.
That's right, we may be losing money, or at least not coming close to our targets, but by golly we are going to double down on our losses like a feverish gambler at the Blackjack table unable to step away in the face of all logic and evidence to the contrary.
And it's not just Bain with these findings. Other firms like Deloitte and PwC have found similar results, so much so that you have to wonder why companies insist on throwing more money at the problem instead of stepping back and maybe figuring out a better way forward. There are best practices, but as the PwC report found, only a small percentage of companies are actually implementing them.
It's the data, stupid
When I started exploring the current generation of AI about three years ago, I remember having a conversation with a consultant, who told me he would go into companies to have an AI conversation, and it would very quickly become a data-readiness discussion.
Believe it or not, that's still the case. Bain found that 41% of respondents cited data access and integration as the biggest single factor holding back their AI progress. So three years later, it's still a data problem, although Bain is careful to point out that the data problem should not be a paralysis problem. You still have to try.
Another big issue that I've heard about continually is using AI to automate a process without looking at it in a new light. If you have the opportunity to remake an inefficient workflow with agents, don't just put an agentic wrapper around the old one. As the report states, "The question to ask before any AI program is approved is not 'Where can we apply AI?' but 'If we were designing this process from scratch today, what would it look like?'" That kind of approach from leadership is more likely to get rank and file employee buy-in because the folks doing the work know those inefficiencies better than anyone.
And even worse, perhaps, is that the finance people are not pushing back about the numbers. Bain recommends looking at returns from past automation efforts and using that as a benchmark. If anything, this generation of AI takes more effort, making it harder to project accurately. "Approving the next program on the assumption that [projected returns are real] without verifying it is the most avoidable financial risk in most companies' AI portfolios right now," the Bain report found.
That's what PwC found as well. Leading companies systematically track business impact and hold senior leaders directly accountable for AI outcomes, sensible business practices it seems that far too few companies actually follow.
It's ultimately about sticking to the basics. As I've written many times, enterprise fundamentals don't go out the window because we have new technology. Instead, they matter more than ever, whether it's prepping the data, reworking a process or trying to understand the underlying financial benefits (or lack thereof). It all comes back to fundamentals, and if you don't follow them you're very likely throwing good money after bad, and shouldn't expect to get a lot out of your agentic efforts.
~Ron
What's new on the blog 📰
Drew Houston has built a legacy any founder would be proud of
The story goes that Drew Houston came up with the idea for Dropbox while on a long bus ride after realizing he had forgotten his thumb drive at home and had no way to access his files.
Nineteen years later, Houston announced he would be stepping down after a transition period to launch a new AI-focused startup outside of Dropbox, and he'll get a chance to see if lightning can strike twice.
Confluent's Jay Kreps returns to his large company roots with acquisition by IBM
I caught up with Confluent CEO Jay Kreps at IBM Think earlier this month in Boston. We chatted about what it was like being back inside a large company after being acquired by Big Blue at the end of last year.
As IBM tries to leverage acquisitions like Confluent to help orchestrate AI, capturing data in real time becomes a critical component for models using data to understand context.
"We bring the ability to do these very rich transformations for processing of that data, and the last mile in the chain is the ability to plug that into these AI models and serve up context off of that."
Aaron Levie's take on how AI moves the work goalposts — and what that means for jobs
I connected with Box CEO Aaron Levie recently on the FastForward on PPN podcast. We covered a lot of ground. I pulled this part where Aaron talks about his theory on why AI actually expands work and increases the need for humans, and turned it into an article.
"What I tend to see is that the work just expands based on the kind of tool capabilities that we have, and I've seen no evidence at any kind of macro scale where that doesn't happen," Levie told me on the podcast.
As developers shift to AI coding, the nature of engineering is changing
AI is inevitably going to have a huge impact on just about every job, but perhaps none more so than the developer role where it is already being widely used.
I spoke to execs at Intuit, Amazon and MetLife to understand just how much the role is shifting and what it could mean moving forward in how we think about engineering.
"From a philosophical change, the skills that are required by the builders, the software engineers, are to really understand software architecture, to know what great software actually looks like, and how you make sure that the coding agent builds software that actually solves the problem, is high quality, can scale, and is not full of defects," Intuit CTO Alex Balazs told FastForward
Why Chen Goldberg walked away from Google to help build CoreWeave’s AI cloud
Chen Goldberg walked away from a prestigious role on the Kubernetes team at Google in 2024 to join neocloud CoreWeave. She saw the opportunity to build the next generation of infrastructure around AI, a challenge any engineer would relish, and she took the leap.
"I joined CoreWeave in August of 2024 with the realization that we were at the edge of yet a new era where infrastructure matters even more than before," Goldberg told FastForward.
News of the Week 📣
Off with their heads: Salesforce buys headless content management company Contentful
Salesforce has seen the future, and it believes the power of its platform is not in the front-end interface, but in the back end where the data resides. That's because agents don't typically need a front end. Being digital, they can interact directly with data in the database.
While Salesforce offered headless products in the past, they went all in when they announced Headless 360 in April. This week the company announced an agreement to acquire Berlin-based headless content management company Contentful, giving the CRM giant its very own back end CMS offering. The terms of the deal were not disclosed.
Here's how Salesforce describe the transaction: "The acquisition will enhance Salesforce’s Headless 360 with a native, enterprise-grade content layer that connects customer data with engaging content experiences across Salesforce’s leading applications."
And as much as that is pure press release gibberish, there is an element of truth to it. In a world where brands need to be able to pull their marketing and sales materials into a personalized experience, this tool gives Salesforce the potential to do that together.
Whether it works or not is another matter, but it definitely is a smart move and if the company executes, it could be a major addition to the Salesforce platform that enhances its usefulness in the age of agentic AI.
Broadcom is probably wondering what it needs to do to impress Wall Street
Broadcom reported earnings this week, and by most measures, it looked to be a fine quarter indeed, but if you make chips and your name is not Nvidia, you may struggle to impress Wall Street no matter what you do.
The company reported $22.19 billion for the quarter. That was up 48% over the prior year, while forecasting $29.4 billion for the upcoming quarter. That sounds pretty good, with EPS beating estimates, revenue coming in just below expectations and guidance topping Wall Street’s forecasts, per CNBC. The result though was not a reward, but a sound beating from investors with the stock down roughly 15% overnight and 12% after a day of trading.
This in spite of AI-related semiconductor revenue coming in up 143% for the quarter — also above forecast. The disappointment came from CEO Hock Tan declining to raise the company’s full-year AI revenue guidance and from Q3 AI chip guidance landing below some of the more optimistic expectations from analysts.
It had five straight quality quarters, per Yahoo Finance, and analyst expectations were stoked by the prior quarters. When the company failed to raise its outlook, Wall Street lashed out, but feels like a pretty good quarter, doesn’t it?
Autodesk acquires building maintenance software company MaintainX for $3.6 billion
It's been a tough year for almost all SaaS software companies in the public markets and Autodesk hasn't been spared, finding its stock down over 20% this year. But it's not sitting idly by while its market cap erodes. Instead it made another move this week, spending $3.6 billion to buy building operations and maintenance software startup MaintainX.
It's a hefty amount of money, but it's a complementary piece for Autodesk's growing construction portfolio. The company intends to launch Autodesk Operations portfolio with MaintainX as a centerpiece if and when the acquisition passes regulatory scrutiny.
As Autodesk put it, "The proposed acquisition of MaintainX is intended to strengthen Autodesk’s ability to connect operations workflows with the broader lifecycle, helping teams make faster, more informed decisions over time."
MaintainX was founded in 2018 and raised close to $254 million including a $125 million Series D last July led by Bessemer Venture Partners, per Crunchbase data.
What I'm reading 📚
I'm an AI Scientist. My Daughter Refuses to Use AI. And I'm Okay With It.
~By Rana el Kaliouby, Ph.D, LinkedIn
Tokens or humans? The new corporate trade-off
~By Deirdre Bosa & Jasmine Wu, CNBC
EU restricts US cloud services plan: key rules and questions
~By Tom Dennis, Raconteur
What I'm watching 📺
Sam Altman: People are right to be anxious about AI
~CNBC Power Lunch
Look who's talking 👄
" So I think this is the most fair contribution – criticism right now of AI, which is, you hear companies saying, I am spending a ton of money on AI. And I know some great stuff is happening, but I know there’s a ton of waste, and you know, when – how long do I have to wait for it to really show up in revenue, and how long do I have to wait to really get the costs under control? And I assume that the industry will figure that out pretty quickly, but I think that is a fair, a fair issue."
~Sam Altman in this week's CNBC Power Lunch interview on lack of ROI on AI.
