FastForward #77: Making AI work is getting expensive
Hi everyone. Hard to believe that the summer is winding down already. I'm working from San Diego this week, hence the later newsletter delivery. If you like what you see, please share this week’s edition with a friend, and encourage them to subscribe. It really helps.💌 Sign up here.
ForwardThinking 🤔
Making AI work is getting expensive
Like so many technologies that came before it, it turns out that AI is more complex to implement than we first thought (or at least were led to believe). Survey after survey finds all but a few notable exceptions are stuck in proofs of concept, unable to scale to the broader organization.
As a result, companies are being told that they need help in the form of the forward deployed engineer role. Broadly speaking, FDEs are engineers who work directly with customers, often embedded inside their organizations, to help solve difficult implementation problems. This is an individual who requires not just technical chops, but also softer skills to help get customers to the next level when it comes to AI. That combination is hard to find, yet everyone from the AI labs to the cloud hyperscalers to the systems integrators needs thousands of these people. Good luck.
Customer success teams are not new by any means. Companies selling sophisticated software solutions have learned that effective implementation requires people who work directly with paying customers to guide them. FDEs are an extension of that function, but with these engineers much more directly involved.
Palantir has used FDEs for years, but its version of the role goes beyond deployment support. The company describes FDEs as owning the customer problem and doing whatever is needed to solve it. The question is whether other companies can adopt that model more broadly, particularly when the necessary skill set is both specialized and difficult to find.
Not only do you need someone with deep engineering ability, today's FDEs also need to understand how to transform complex workflows into agents, combined with a deep understanding of agentic AI itself, all while having the people skills and patience required to work closely with customers for long periods of time.
Then consider you might also need a very specific domain understanding like insurance or healthcare and it raises the degree of difficulty even further. It's also expensive. A recent OpenAI posting for an FDE cited a salary of $162,000-$280,000 plus equity.
The talent problem
The big companies claim they need thousands of these people. Where exactly are they going to find them? OpenAI is asking for five-plus years of customer-facing engineering experience and traveling up to half the time, a profile that could limit the candidate pool even further.
If you put people in place without the necessary skill set, companies are going to balk at paying for an inferior service, especially when they are already paying these companies big money for the AI tooling they are having trouble implementing in the first place.
This comes against a backdrop of companies already admitting they are over-spending on AI. A July report from McKinsey found that 93% of respondents had already exceeded their AI budgets, a stunning admission (especially when you consider the survey was conducted in May), with 46% over by between 10 and 30%. And believe it or not, it's about to get worse with a majority of respondents reporting they expect AI spending to increase by at least 25% in the next 12 months.
It would seem that CFOs have to put on the brakes at some point. You can't just keep throwing money at the problem. If you are lucky enough to get the most elite people to help your organization succeed, then it may be worth the additional cost.
The question is whether there are enough people with the necessary technical and business skills, and whether companies can afford to deploy them at sufficient scale to justify the additional spending. If not, FDEs could be adding cost to a process that is increasingly out of control.
~Ron
What's new on the blog 📰
How a 150-year-old insurance company is moving into the AI era
I interviewed MetLife CIO Nick Nadgauda to learn how a 150 year old insurance company is transforming its tech stack for the AI age.
Using COBOL programs as an example, he says he looks at what to transform more from a maintainability perspective than a strict modernization one.
"I don’t necessarily have a problem with COBOL. The real problem in my mind is when you’ve got systems that you can’t support, where you can’t find programmers, you can’t find talent or they’re hard to modify."
Why Cisco’s Jeetu Patel thinks product strategy should drive acquisitions
I had the pleasure of hosting Cisco president and chief product officer Jeetu Patel on my FastForward on PPN podcast recently. I asked Jeetu about his build versus buy strategy, and he told me it was all about the products.
"The reason I don't like having a strategy which says what is your acquisition strategy, is that I don't have an acquisition strategy. I have a product strategy,” Patel told FastForward on PPN.
The entire podcast is embedded in the article.
IBM and OpenAI hope their partnership is the beginning of a beautiful friendship
IBM's partnership with OpenAI may be a no-brainer, but it should help both companies.
Note that I originally wrote about this partnership in a LinkedIn post, then expanded that in a News of the Week piece in FastForward # 75. This analysis takes a deeper look.
The repurposing of content across different platforms is intentional and part of my ongoing strategy to reuse content wherever possible.
Read the full story>>
In the debate over open-weight models, Capital One makes a different case
There is a battle raging in tech over the use of open weight models. On one side, the AI labs, the same ones telling us that their models got loose and hacked several websites, say the open weight models are unsafe.
Other companies argue that they need to be able to customize models, and the proprietary ones from those same AI labs make that much more difficult to do.
I spoke to Milind Naphade, SVP of AI foundations at Capital One, whose company doesn't care about the debate. They just need the flexibility that open weights provide to take advantage of their data.
"To do all that needs open-weight models, we just cannot do that without them. And why is that important? Because that's the only way that we can fulfill our twin mandates of delivering high accuracy with regulatory compliance," he said.
AI is rewriting the CISO job description
In the final part of my 3-part video interview series with Zscaler's Swamy Kocherlakota, we talked about how AI is changing the role of the CISO.
"So, in my mind, the CISO now has to evolve to become much more application-centric, data-centric, above and beyond the infrastructure centricities that they have. So, there's a new breed of CISOs that will come in as a result of this," he said.
News of the Week 📣
Salesforce-Anthropic partnership could be big deal for both companies
Salesforce announced a major expansion of its partnership with Anthropic this week. The partnership, which is being called Claudeforce (Claude and Salesforce, get it? ), enables Salesforce customers to access Salesforce data from inside Claude without having to use the Salesforce interface, a major step forward for the SaaS giant.
The first part of the partnership involves 37 pre-built sales skills built into Claude, effectively making Claude the interface for Salesforce data, workflows, business logic and prebuilt skills. The partnership also calls for embedding Claude into Slack, the enterprise communications tool Salesforce bought in 2021 for $28 billion.
Jason Andersen, an analyst with Moor Insights & Strategy likes the partnership and sees it as an extension of the Headless 360 enhancements announced last week (and which we covered in FastForward #76).
"I think it’s great. Salesforce is very serious about what it is calling its headless strategy," Andersen told FastForward. "The partnership with Anthropic is a great proof point that they are serious about making agents a first class citizen in their ecosystem."
For Anthropic, the announcement comes ahead of its impending IPO, and for Salesforce, it comes at a time when investors are wary about the economics of the per seat license model. This partnership, and likely others like it, means the company is showing it can move beyond a pure SaaS licensing strategy moving forward as the interface takes a back seat to the data being collected inside the system.
It's worth noting that Salesforce stock jumped 22% the day after the announcement, its second highest one-day jump ever, per CNBC, showing that investors like it very much indeed (along with some other aspects of this week's earnings report).
Salesforce still has to show it can expand this partnership, while making it part of a broader partnership strategy, but it's a positive initial step in terms of making the data be the key part of the platform.
Nvidia just keeps on growing with no sign of slowing down any time soon
Ho hum. Nvidia announced another monster quarter yesterday afternoon with revenue of $96.2 billion, up 106% over the prior year. As though that weren't enough the company also announced impressive guidance for the upcoming fiscal year, a double win.
CNBC reported that the GPU giant expects revenue to grow 70% next fiscal year, while analysts were predicting 44%, a pretty insane gap that shows that the demand for GPUs is expected to continue for some time. This comes as CFO Colette Kress says that the guidance reflects supply constraints. In other words, the growth could be even bigger if they could just produce the chips fast enough to meet demand.
"The surge in AI demand is driving a global infrastructure build-out, supported by an expanding and diverse set of growth opportunities, spanning hyperscalers, AI labs, AI natives, enterprises and sovereign customers," Kress said in the earnings call with analysts.
Nvidia stock was up 7% over night on the news, rather modest considering just how good the report was.
But it's not necessarily all sunshine and light. As Vivek Arya from Bank of America asked, there is the issue of circular investments and the fact that the AI labs and hyperscalers are all making their own chips, which could eventually have an impact on Nvidia earnings.
Not surprisingly, CEO Jensen Huang defended the company, saying the Nvidia platform spans the entire AI lifecycle, while custom XPUs produced by the hyperscalers and AI labs tend to be workload/cloud-specific. As for those investments, he sees it as a once-in-a-generation opportunity and has no regrets over his company's decision to invest in these AI labs, other than wishing he invested sooner when it was a better value.
Harness is redesigning the code repository for AI speed
AI coding is increasing code production velocity in a dramatic fashion, so much so that GitHub ran into a capacity problem earlier this month that shut down big parts of the service for almost 8 hours (as we reported in FastForward #76).
Harness, a software delivery platform aimed at software engineers, is trying to solve the velocity problem by redesigning the system for storing and reviewing code.
The company claims that the updated code repository is designed to handle larger volumes that AI coding brings, while working with both humans and agents. The repository works in conjunction with the new AI Code Review tool, which can use information from the broader software delivery pipeline to check for issues faster, partly based on past incidents and problems via the SDLC Knowledge graph.
The idea is to build a system that can keep up with volume and speed, with both humans and agents accessing it via MCP and the command line, while using information from the broader software delivery pipeline to help understand where possible issues may be hiding.
Both tools are available now.
What I'm reading 📚
How the Marks & Spencer Cyberattack Revealed a New Era of Social Engineering
~By Keegan Henckel-Miller, DeleteMe blog
Dr. Dre and Jimmy Iovine Think A.I. Is Good for Music
~By Jordyn Holman, Wall Street Journal
Is it legal to train AI models on copyrighted books? It’s complicated
~By Amanda Silberling, TechCrunch
What I'm watching 📺
Why Technical Founders Hire Marketing Too Late
~Walter Thompson, Fund/Build/Scale
Look who's talking 👄
"So every time there's a step function improvement in AI, there's a human bottleneck that emerges simultaneously. And that in my mind is a state of perpetuity."
~Jeetu Patel, president and chief product officer at Cisco on the FastForward on PPN podcast this week.
