Atlassian and Expedia see experimentation as a key part of learning about AI

There is an expectation out there, due in large part to the hype around AI, that companies are rolling it out widely and seeing meaningful results. Yet research suggests that most organizations still struggle to scale AI projects beyond pockets of success.

At a panel at HumanX in Amsterdam last week, called Building for the AI Era, I spoke with Avani Prabhakar, chief people and AI enablement officer at Atlassian and Chris Burgess, global VP of technology and deputy CIO at Expedia. The two companies are not only examples of organizations trying to implement AI broadly, they are also partners, which created  an interesting dynamic on the panel.

What stood out in particular was that both companies are still experimenting widely with AI, even as they embrace the technology across their products and their own organizations.

Prabhakar says it's not as though there is a state of being done, and that's where the notion of ongoing experimentation comes in. "I feel like the speed of change is so fast," she said. "The only way to learn more about AI is just by creating faster learning loops internally, and that's what we are seeing."

Burgess is taking a similar approach at Expedia, specifically looking at how they can help employees be more effective, more productive and move faster. It's a lofty goal, and it didn't just happen. It involved working with companies like Atlassian to push Expedia employees to use these tools.

"If we go back eighteen months, we didn't have a lot of tools in the environment. We didn't enable the capabilities within the existing platforms, then we partnered with companies like Atlassian where people were already doing their work to enable those tools and encourage experimentation."

He said by looking at adoption rates, the company was able to measure success, at least initially. Today, they continue to experiment, while focusing more on tangible business results. 

What's working?

They are seeing pockets of productivity increases. But once they find some success, they try to get the people using the tools to evangelize and teach others.

"Every single function is slightly different, and we've brought together essentially a group of super users that we call pioneers internally," Burgess said. "We look at what they're doing and we have them partner within the different functions." That could be within or across functions, depending on the situation, but the idea is to get people who are using these tools, and seeing results, to help others.

Not surprisingly, both companies have seen real productivity gains for developers using AI coding tools with Burgess reporting some tasks improving from weeks to days compared to before using AI tools.

But Prabhakar says there are lots of aspects of the software engineering job that AI isn't solving right now. "If I talk about left of code before prototyping and if I talk about right of code in terms of maintaining the code, we haven't really seen AI kind of really increasing the output. So it's mostly in the code gen space for software developers."

In terms of non-technical users, she says it's best to give them a goal, a problem to solve, and then ask them to use AI to help, rather than pointing to a model and asking them to have at it. She gave the example of an onboarding agent created by the HR team, which has little technical expertise compared to the developers.

"For HR folks, employee onboarding has been one of the pain points for them at any given point in time. It was the talent team that went and built an agent by themselves. They learned a lot, but because they were the users, they knew the pain points," she said.

The experiences of Atlassian and Expedia suggest that experimentation isn't necessarily a stage companies need to get through on their way to implementing AI at scale. With the technology continuing to change so rapidly, figuring out what works, learning from it and adjusting along the way as needed may simply be part of the process.