FastForward #81: AI still has a major trust problem.
Hi everyone. Can't believe it's fall and the leaves are starting to turn outside my office window already. I’d like to thank Akamai once again for supporting FastForward for the last four issues. If you're interested in advertising, please drop me a line at ads@fastforward.blog. If you like what you see, please share this week’s edition with a friend, and encourage them to subscribe. It really helps.💌 Sign up here.
ForwardThinking 🤔
AI still has a major trust problem
But something is happening here
And you don’t know what it is
Do you, Mr. Jones
~ Bob Dylan, Ballad of a Thin Man
This week, OpenAI and Meta released competing versions of a new kind of persistent agent for business. To work well, each needs sweeping access to your company's data. And that's the rub, isn't it?
Both companies have a major credibility problem. The more sophisticated these tools become, the more access they require. The question is whether individuals and businesses are willing to hand that level of control over to companies that have done little, so far to show they deserve it.
If this software is going to be an integral part of businesses, big and small, they have to be able to trust it to be safe, secure and reliable. And right now, the entire industry has failed to make the case, and that's true on so many levels.
Trust is a tough thing to come by these days
I spent last week at the HumanX conference in Amsterdam, where I moderated three panels and heard the European perspective on AI. One message came through clearly: regardless of what the AI labs say or do, businesses need guardrails and security before they can trust AI tools enough to deploy them widely.
Stefan Weitz, HumanX co-founder and CEO, speaking at the event’s opening keynote went so far as to say that without governance, you’ll never get out of the pilot phase (which could explain why so many companies are stuck there).
"AI without governance is simply a toy. If there's no governance in your AI strategy, what you have is not a strategy, it's a pilot,” Weitz said. “And it will never scale because you don't know how you did it, you do not know why you did it, and you don't know who's responsible when it goes wrong."
Swami Sivasubramanian, vice president of agentic AI at AWS, also speaking at HumanX, said we tend to skip governance because it's unglamorous, and want to rush to the glory of the final result, but that’s exactly what can trip us up. “We skip verification because it slows us down, and we actually skip governance because it's not as exciting as the next feature. This is exactly what's happening with AI too,” he said.
He's right of course, in spite of knowing better, companies (and individuals) sometimes skip over the boring parts, but governance is a key ingredient to AI success. "The question is whether you have the rigor and the discipline to go execute it," he said.
To get there though requires trusting the software and the companies that built it, and there's a real gap there right now. AI projects need guardrails to ensure they are doing the job you built them to do. Without them, projects will flounder, and you may find yourself, like the AI labs, admitting to harm you never intended and could have prevented with the right safeguards in place.
~Ron
What's new on the blog 📰
Glasswing's Rudina Seseri says AI has changed the formula for startup success
When I spoke to Glasswing Ventures' co-founder and managing partner Rudina Seseri at TechCrunch Early Stage in 2024, she was already seeing how AI was changing the investing landscape. Today, she says that has become even more pronounced.
"You know, even two or three years ago, if a company went from zero to $1 million in ARR in its first year in the market, you'd be like, I have a unicorn in the making. Now that number looks more like zero to $2.5 million," Seseri said.
Atlassian and Expedia see experimentation as a key part of learning about AI
Last week at HumanX in Amsterdam I had the privilege of interviewing Avani Solanki Prabhakar from Atlassian and Chris Burgess from Expedia on a panel about their company's AI journeys.
A big part of it is understanding that with the rapid rate of change, experimentation is built into the process, and you are never really done.
This quote from Prabhakar kind of sums it up:
"I feel like the speed of change is so fast," she said. "The only way to learn more about AI is just by creating faster learning loops internally, and that's what we are seeing."
As Adobe embarks on a new era, it pushes deeper into AI-powered customer experience
When I interviewed Amit Ahuja, Adobe's SVP of CX products, it was before we knew his boss, Anil Chakravarthy, would be named CEO of the company, demonstrating the company's commitment to the enterprise side of the house.
As Adobe transitions to a new era, creative and enterprise need to work together, and change the investor narrative that Adobe (and other SaaS companies) are less relevant in the AI era. As Ahuja told me, it's not like Adobe just jumped on the AI bandwagon when ChatGPT came out. It had been using prior generations of AI for many years:
"Even before Sensei, we've been using kind of general purpose AI for ages here at Adobe, and we've been doing that obviously across Adobe, whether that's on the creative and professional side, whether it's on a document productivity side, or whether that's on the experience orchestration side."
AI has forced Harness to rethink the roadmap concept
When Harness founder and CEO Jyoti Bansal appeared on the FastForward on PPN podcast recently, he talked about how the speed of AI coding had forced him to completely rethink the notion of a product road map.
"Like if you want to build something, just build it. Why is it sitting on the roadmap somewhere," he asked.
The entire podcast is embedded in the article. Well worth a look.
AI is making scammers smarter and helping Visa fight back
I spoke to Visa chief risk and client services officer, Paul Fabara about the challenges his company faces trying to root out fraud from the massive payments network.
One of the interesting things about AI tools is that both sides have access to the same models, but Fabara says Visa has a huge data advantage over bad actors:
"One of the things that crooks don't have access to is the same level of data that we do. And so that makes a tremendous difference when it comes to looking for that fraudulent transaction," he said.
News of the Week 📣
UserTesting is now Auros as company bets on the value of human data to AI
For twenty years, UserTesting has been helping brands by interviewing a growing network of real people to help companies understand how their products and experiences are resonating with customers. It turns out in the age of AI, that kind of data is invaluable.
This week, the company decided to reposition itself to focus on that value, and change its name to Auros. CEO Eric Johnson says that while the UserTesting brand and product names will remain in place, at least for the time being, the more recent AI use cases warranted a new brand to reflect the broader AI-related capabilities, and a changing target buyer.
"We started to see the challenges when we wanted to talk to these executive buyers, and we have been seeing a really good, fast-growing opportunity in AI training over the last probably six months, and we felt like we needed a brand that better represented our vision and our future," Johnson told FastForward.
The company's major asset is its access to millions of humans and the feedback they provide. It creates a moat that a lot of companies would envy. "Our moat is that we have 7.6 million people who are willing to provide feedback on a wide variety of experiences, and whether those are product experiences, whether those are some sort of marketing experience, whether that's providing feedback to train agents, that 7.6 million person network with by far the lowest fraud in the industry is the hardest thing on earth to create," he said.
Liz Miller, an analyst at Constellation Research, agrees that the Auros network and the data and insights created from it have tremendous value when it comes to training models and giving them real-world grounding. "AI is steeped in expertise but lacks the judgement and experience of humans," Miller said. "This is why tapping into human networks to not just train, but to validate and confirm that our AI intentions are meeting our very human needs becomes a critical step that has been, until now, an imperfect science based on guessing."
Editor's note: I moderated a paid event for UserTesting in 2025. FastForward always maintains editorial independence and follows my published editorial guidelines.
AMD is acquiring Fei-Fei Li's World Labs for $8.2B as chip maker makes big talent push
This week chip maker AMD made a surprising announcement: It intends to acquire World Labs, the startup from AI pioneer Fei-Fei Li for an astonishing $8.2 billion, just two years after it launched.
The company has focused on spatial intelligence, which could have increasing utility in robotics, but the focus of the acquisition might have been more about acquiring Li and her top-notch team of AI experts, as much as anything else.
AMD CEO Lisa Su certainly recognized the value of Li and her team. "Fei-Fei and the World Labs team bring exceptional research leadership and model expertise. Together, we can use that insight to develop the hardware, software and systems that will power the next generation of AI and strengthen the open AI ecosystem.”
Li sees a growing relationship between AI and compute and this acquisition should help accelerate that. "The research and technical breakthroughs we have achieved since our founding in 2024 have given us a clear vision for AI’s potential to solve problems in the spatial and physical world," Li wrote in a blog post announcing the deal. "To accelerate into this future requires scaling our efforts, scaling our reach, and getting closer to the hardware.”
It's not a coincidence that the two companies have joined forces. They began working together last year in what Li called "a deep technical partnership."
The company has raised over $1.2 billion, per Crunchbase. The most recent funding was a $1 billion Series B earlier this year. While the company did not release a valuation, unconfirmed reports pegged the value at $5 billion. Crunchbase lists multiple investors including VC firms like Andreessen Horowitz and NEA, corporate venture arms like AMD Ventures and Autodesk and individual investors like Marc Benioff and Eric Schmidt.
The deal, which is subject to normal regulatory review, is expected to close by the end of the year.
Salesforce to acquire Listen Labs as it builds out its agentic customer stack
Salesforce is getting back to its acquisitive old self. This week, the CRM giant announced that it intends to acquire Listen Labs, an AI-powered customer research platform. While the two companies did not reveal the price, several published reports pegged it at around $2 billion.
It comes on the heels of the Fin acquisition, a $3.6 billion deal that was announced in June and closed last month. That the deals came close together shows that Salesforce could be attempting to build a soup-to-nuts agentic customer stack that includes both customer research (what they are thinking) and customer actions (what they are buying and contacting customer service about after they do).
Put the two together and it could provide Salesforce with a lot of raw data about customers that could be put to work to give agents more accurate context when dealing with customers. Agents are increasingly handling interactions that previously went to humans, at least in lower-leverage situations.
All of that is the possible vision. Salesforce hasn't made that connection yet publicly as far as I can tell, but if you connect the dots and read the tea leaves (especially when you look at the story earlier about the new Auros brand), it makes a lot of sense.
Listen Labs was founded in 2023 and raised approximately $96 million along the way. The company was reportedly looking at a $125 million Series C at a $1.5 billion valuation when Salesforce came a-callin'.
The deal is of course still subject to regulatory approval, but Salesforce expects it to close by the end of January 2027.
What I'm reading 📚
I created an interactive digital avatar of myself — and you can talk to it
~By Dominic-Madori Davis, TechCrunch
Wikimedia's CEO on the AI traffic armageddon
By Ina Fried, Axios
The AI Bros are Partying
~By Mara Hoplamazian and Emily Kennard, The Washington Sun
What I'm watching 📺
The SaaSpocalypse that wasn’t with Atlassian CEO Mike Cannon-Brookes
~Decoder Podcast with Nilay Patel
Look who's talking 👄
"The alternative to governance isn't speed. The alternative to governance is 40 pilots in 40 business units, none of which can be defended to a regulator, an auditor, a customer or a court. And every single one of them dies the second somebody asks a hard question."
~HumanX CEO Stefan Weitz at the HumanX Amsterdam keynote last week.
