When it comes to AI, IBM heads for its happy place in the middle

Few tech companies have survived as many platform shifts as IBM, from mainframes to PCs to the cloud, and its next big bets are on AI and quantum computing. Last week, the company held its Think customer conference in Boston, where the message was clear: AI is hard, and IBM wants to help.

It had to be a comforting message when the research suggests that most companies are struggling to implement AI successfully. In a world of AI FOMO, where everyone wants AI but few are getting it right, it probably was reassuring to know they were not alone. It's sort of like high school where it seemed like the cool kids had all the answers, but they were just as confused as the rest of us.

So how can IBM help? It wants to be the neutral middle layer, positioning itself to solve all those pesky problems we've been talking about around data, governance, security and integration. These are the hard parts that the SaaS vendors, AI labs and the cloud hyperscalers tend to gloss over. They are the ones you can't vibe code your way through. 

CEO Arvind Krishna suggested the reason companies were struggling with AI was the plumbing problem, an area he believes his company can actually address. "They don't fail because of the AI [technology]. They often fail because of what's underneath: siloed data, fragmented infrastructure, multiple clouds with no coherent operating models. The models don't really matter unless the foundation is correct," Krishna said in the keynote.

He believes that hybrid, a mix of cloud and on-prem resources, is the answer, and it's a particularly convenient answer for IBM. The company is betting that it can help customers build AI regardless of the infrastructure they are using, then combine that with internal IBM tools from the likes of Red Hat, HashiCorp and Confluent, while mixing in tried and true services and consulting to push customers along.

A matter of trust

In a world where other companies are building the major models and the cloud vendors own the infrastructure, being a neutral orchestration layer makes sense for IBM. Krishna teed it up this way: customers can't trust companies with a dog in the fight to be true partners, especially given the circular nature of some of these relationships. "What is the chance an enterprise is going to trust a domain specialist to be the control tower for other domains which are buying from other vendors, who all then compete with each other," Krishna wondered last week.

He acknowledged that if a company was using a single cloud vendor, it makes sense to get all the underlying services from that vendor, but the vast majority of customers don't do that. That hybrid reality is central to IBM's thesis. "Hybrid is not a compromise. Hybrid is the architecture that helps give you resilience," Krishna said. He believes that if your infrastructure is spread across multiple environments, you're probably safer from a reliability standpoint, but you still need someone to make it all work together, and that's the management problem that plays into IBM's primary selling point.

It's a role the company has played before, going back to messaging middleware and service-oriented architecture in the early 2000s, and it believes it can repackage it for the AI era by building a unified set of requirements across different agent providers. "I think that's the role that is natural to us that we can play successfully, as opposed to trying to be the provider of the unique agent technology."

So how does IBM bring all of that together into an orchestration stack? It has made three major acquisitions over the years that help, starting with Red Hat, which provided enterprise Linux, Kubernetes, containers and microservices. Then they added on HashiCorp to offer infrastructure automation on top of the cloud pieces in Red Hat. Finally, Confluent brings real-time streaming data, which is particularly important when it comes to putting AI to work across applications.

Sitting on top of all of that is watsonx Orchestrate, IBM's new agent control plane, which is designed to make agents work together, regardless who built them. "We have put AI into the enterprise, whether that is using Orchestrate to connect multiple sources, bring agents from different places, and provide a control plane and governance," Krishna said.

Harder than it looks

While customers at a conference like this are usually there to tell a positive story, if the story is about orchestration, it is worth hearing the flip side. You want to understand what happens when you do not have good governance or clean data, and how that can affect the quality of the agents or the answers you get from the AI, depending on the use case.

Not surprisingly, the customers were people who believed in the AI vision and gave the standard kind of feedback that the tech is ready, it's the people and systems that aren't. I have argued that AI isn't as ready as industry leaders would like us to believe. It was a tension that leaked through consistently when customers spoke.

Elevance Health chief digital information officer Ratnakar Lavu, acknowledged that when it comes to building agents around internal processes, you can't just put lipstick on a pig and expect it to look better. You have to rethink the process first, then apply AI. "Technology is one component of it, but it's not about technology itself. I start with the experiences, and then underneath the experiences, you have to fundamentally change processes, and the process change has to be powered through technology." But the change management around getting people to think that way is extremely challenging and it's an area where a lot of organizations are still struggling. 

EY's tax platforms leader and chief product officer Christopher Aiken went further, saying the technology is no longer the barrier at all. "The technology is ready. It works, and it works well. And in the AI space, that's saying quite a lot. One of our bigger challenges is around leadership." What he meant is getting the organization pointing in the same direction when it comes to AI implementation, and that's not always easy to do. If it's not done right, it can leave a lot of confusion in the rank-and-file employees.

IBM didn't focus strictly on present market needs, however. It also talked a lot about quantum computing, an area where it believes it has a big advantage and is seen as the frontrunner, at least for now. Krishna sees quantum and AI playing nicely together.

"Quantum and AI, by the way, do not compete. They converge and they complement each other. Quantum can help uncover what AI cannot yet compute. Then AI learns from the quantum and you can make faster and faster progress on algorithms and on computations," he said. IBM consistently sent the message that it sees its quantum advantage bearing fruit in the coming years.

IBM is making this bet under pressure from investors who have watched the stock underperform for years, while also fighting for a sustainable position in a vast, crowded market where every big tech vendor and consulting giant is still figuring out what works. If it cannot convince customers to buy in on its AI orchestration solution, it will be forced to lean heavily on a quantum roadmap that, while ambitious, still has to be proven at scale.