The AI worked fine. The Pilot “Failed”

Imagine classic expectations for a successful pilot.

The demo impresses everyone. The team proves that the technology can do what they hoped it could do. It gets approved to move forward.

Six months later, nobody is using it. We make all sorts of excuses. The technology wan’t ready for ‘prime time.’ The pricing was wrong. Perhaps we blame the vendor (always convenient). And of course we had conflicting priorities. We always do.

All true. At least in part. But the excuses leave out the cold hard reality. Change is difficult, and is always more complex than anyone expected. More importantly, perhaps, is that change comes with a cost, and not everyone was bought in or was able to bear that cost. Things got stuck.

You might think that I’m skeptical about doing pilots from all of this. Quite the contrary. I love pilots. I actually believe pilots are the best way to reduce risk and advance innovation. I say so in my book, Stone Soup Strategy.

So what are we missing?

We tend to do pilots for the wrong reasons. We pilot the happy path, so everything looks good, and ignore the difficult risky part, that might lead to failure. And we end up with one more implementation that no one actually uses. Let’s pick an example to make this more concrete.

Suppose you’re considering using AI to automate part of a customer-service process. You build a prototype, feed it some representative data, and demonstrate that it can generate good responses.

Great.

But can employees incorporate it into their workflow? Do they trust it? Who is responsible when it produces the wrong answer? Can you access the necessary production data? Does the economics still work at scale? Will customers accept it? What changes to the existing process are required?

THERE ARE A LOT OF QUESTIONS TO ANSWER.

And these questions span people, departments, workflows, compliance, etc.

No wonder things get stuck, even when they work. If the pilot succeeds, is anyone prepared to make the decision to proceed?

Pilots need a champion. Furthermore, a good pilot should be designed around uncertainty, not technology. The most important question is “what don’t we know?”

Before building anything, identify the value proposition, the desired outcomes, and the decisions that need to be made.

I explore this in my Maven course, The Pilot Trap — Why AI Pilots Fail.

The central idea is simple:

Don’t start an AI pilot by asking what you can build. Start by asking what you need to learn.