Avoid AI Regret

We are starting to see the results of early AI experiments in enterprise and not surprisingly, they’re not amazing.  A recent study by Orgvue says that 55% of enterprises regret decisions to replace people with AI for example.  Of course, by some estimates up to 65% of all IT projects fail, so AI is not actually worse than other technology areas.  It’s not much better either.  Disappointingly, there is no magic in the world.  Every single tech decision—every piece of software, every hardware upgrade, every shiny new AI tool—needs to start with a specific problem, aim for a defined outcome, be measured, and then iterated upon.

This isn't just my soapbox; we've seen the same pattern before with things like cloud adoption. And we're seeing them again with artificial intelligence, including the emerging concept of agentic AI, where software acts on your behalf. The hype is incredible, but the rubber hits the road when you ask: "What is this actually doing for the business?" and “How do we measure success?”

Remember that report from Orgvue on AI and workforce transformation we talked about? It laid out some pretty stark realities that underscore exactly this point. The initial, perhaps overly enthusiastic, dive into AI by some organizations is leading to some hard lessons learned.

Here's what those findings tell us, screaming the need for a solid business plan:

• The Regret of Rushing Redundancies: The report highlighted that a significant chunk of leaders decided AI made some employees redundant, only for more than half of those to regret the decision. This isn't just an HR issue, it's a business failure. It means they didn't fully understand how AI would integrate, what roles were actually impacted, or what the ripple effects on productivity, morale, and institutional knowledge would be. That's a direct consequence of not starting with a clear business benefit and a detailed plan for achieving it. Were those redundancies genuinely necessary to achieve a quantifiable business goal, or were they based on a premature assumption?

• The Skills Gap Isn't Closing Itself: Despite pouring money into AI, organizations are realizing they don't have the internal skills to make it work effectively. Leaders are boosting training budgets and seeking external help. This proves that deploying the tech is only step one. The business benefit doesn't magically appear; it requires people who know how to leverage the AI to improve workflows, analyze data, or interact with customers. If your AI strategy doesn't include a workforce strategy focused on skill development, you won't capture the value.

• Lack of Clarity on Impact: Many leaders simply don't have a good grasp of how AI will truly affect their business or specific roles. They can't identify which jobs will benefit most or which jobs are most susceptible to automation. This is particularly true for more complex applications like agentic AI, where many leaders admit they don't know how to implement it effectively. Without understanding the how, you can't define a meaningful business goal or measure success. It's like buying a complex piece of machinery without knowing what product it's supposed to help you make. 

Think about an AI agent. At its core, it's a tool designed to perform specific tasks. Successfully integrating it is akin to hiring a very specialized employee. You wouldn't just hire someone and tell them to "go be productive." You'd give them clear objectives, define their responsibilities, provide training, and set up ways to measure their performance. Does the agent writing first drafts save your team time (a business benefit)? Does the agent managing customer inquiries actually improve satisfaction scores?  Is the quality as good or better than if a human did the work?  How do you know that?  How often have you reviewed and iterated on the solution?

The trends show that AI is being adopted and delivering value in specific areas. We see examples like virtual assistants handling billions of customer interactions, content creation tools being used hundreds of millions or billions of times, and specialized AI software driving significant revenue growth in industries like financial services. These successes likely stem from identifying specific problems these tools can solve and measuring the results across multiple iterations. 

The Orgvue findings are a cautionary tale against the alternative—deploying AI blindly, chasing the hype without a grounded business case. Businesses focused purely on simplistic cost-cutting through premature layoffs, without a deep understanding of AI's role and the necessary workforce adjustments, are encountering regret.

The real, sustainable value from AI, or any technology, comes from strategic integration tied directly to achieving specific business outcomes. Every single technology investment must pass the "So what?" test. So you have a new AI tool? So what does it do for the business? Does it increase revenue? Reduce costs? Improve efficiency? Enhance customer experience? Enable innovation?

Stop treading water with generic deployments that offer little competitive advantage. Instead, focus on identifiable business problems where AI offers a unique solution that traditional software can't provide. Work with small, focused teams, tackle manageable problems, define your desired outcome upfront, and measure everything.

The technology landscape is constantly evolving, with new models and techniques emerging rapidly. Relying solely on rigid, centralized evaluations might mean you miss opportunities. Flexibility, focused experimentation, and quick iteration based on measured business results are key.

And I'll say it one more time, because it's that critical: Start with the business goal and work backwards towards technology. If you don't, you're not investing in the future; you're just setting money on fire.