The AI hype problem: extraordinary results that nobody can check

Boards have rarely been asked to fund a technology on so much enthusiasm and so little evidence. Seventy-five percent of the chief executives in Gartner's 2027 CEO Survey, published on 20 October, plan to increase technology investment. Forty-four percent of the same group struggle to demonstrate a financial return from AI to their stakeholders.

The gap between those two figures is filled, in large part, by claims. Social feeds and vendor case studies carry a steady stream of extraordinary AI results: functions automated in a weekend, headcount cut by half, revenue multiplied. Jay Janes, who founded the AI consultancy Reformat Labs and spent most of his earlier career selling technology into businesses, regards most of those results as untestable, and he argues that their effect on investment decisions is corrosive.

"Influencers and some vendors post and publish extraordinary results, but they are mostly unverified," he says. "They almost always leave out the how, where, when and why of the implementation. That's why we call it AI hype."

The anatomy of an unverifiable claim

Janes does not name the individuals or companies involved. His critique is structural, and it rests on three conditions that he says allow the claims to persist.

The first is incentive. Influencers monetize attention, courses and consulting leads, and a modest result attracts none of them. Vendors and their resellers are rewarded on licenses sold, and the commercial relationship typically ends long before anyone measures what the license returned. Janes says a vendor books its revenue when the contract is signed, and whether the customer ever sees a return is measured months later, if anyone measures it.

The second is selection. A published result tends to be the best outcome from the most favorable deployment, presented without the failed pilots, the abandoned use cases or the hours of configuration that preceded it. Without the denominator, an executive has no way to judge how likely the result is to be repeated.

The third is the absence of accountability. A results claim on a social platform is audited by no one, and an unverifiable number carries no cost to the person who posts it. A finance director who presented a business case on the same basis would not get it through a board.

The omissions Janes lists are precisely the variables a decision-maker needs. The "how" covers the process that was changed and the method used to change it. The "where" covers the function, the sector and the size of the organization. The "when" covers the timescale, including the weeks before go-live. The "why" covers the problem the deployment was meant to solve. Strip those away and a result cannot be compared with anything, least of all with a board's own first project, which will almost always look poor beside it.

What the independent evidence shows

The research available to executives describes a far more uneven picture than the claims do, although each study carries its own caveats.

Gartner's survey covered 203 chief executives and senior business executives in April and May 2026, and the firm sells research and advisory services to the companies it surveys. Its analysts locate the shortfall inside the business. "The AI value problem will not be solved by simply adding more AI to existing ways of working," David Furlonger, a distinguished vice-president analyst at Gartner, said in the release. Asked what would accelerate financial returns from AI, 63% of the CEOs named radically redesigning the operating model.

Boston Consulting Group reached a similar conclusion from a larger sample. Its October 2024 survey of 1,000 CxOs and senior executives across 59 countries found that 74% of companies had yet to show tangible value from AI. The companies that did generate value followed what BCG calls a 10-20-70 rule, putting 10% of their resources into algorithms, 20% into technology and data and 70% into people and processes.

MIT's Project NANDA reported a starker figure in its 2025 study, The GenAI Divide, which found that 95% of the organizations in its research saw no measurable return from generative AI initiatives. The study is preliminary and its sample is narrow, and it is better read as a signal than as a failure rate. Its explanation is consistent with the other two: the authors attributed the gap to brittle workflows and poor fit with daily operations rather than to the models. They also found that more than half of corporate AI budgets were directed at sales and marketing, while the strongest returns were reported in back-office functions.

All three studies place the difficulty where Janes places it: in implementation, in process and in people, the parts of a deployment that published claims tend to omit.

Why the first deployment takes weeks

Janes's approach begins with a narrow selection test. The first candidate for automation is a small, manual, repetitive task that the business would be glad to be rid of, and the task owner's frustration is part of the qualification.

"It has to be something the business or the task owner is passionately frustrated by," he says.

The reason is the effort that follows. In Janes's experience a first deployment takes weeks, and he describes four stages that fill them. The first is mapping the process as it actually runs, including the exceptions that were never written down and exist only in the habits of the people who do the work. The second is testing against real cases, edge cases included, until the output can be trusted without line-by-line checking. The third is people: the team that owns the task has to trust the new method and adopt it, and a deployment that the team routes around produces no return. The fourth is review and tuning after go-live, before the approach is extended anywhere else.

Without a task owner who wants the problem gone, he argues, the project stalls somewhere in those weeks. The processes concerned are usually long-established and material to the business, which rules out haste.

"These are usually long-standing business processes that are important, so you don't want to rush through them," he says. "There is no overnight success with AI."

That timescale is the clearest point of departure from the claims. A result announced as a weekend's work implies a deployment with no mapping, no testing and no change management, which is either untrue or a description of a process too trivial to matter. BCG's 10-20-70 rule puts most of the effort in the same place, in process and people.

The compounding case

The returns Janes reports are modest, and he does not present that as a weakness.

"Most returns, in my experience, are modest, but they are absolutely worth doing," he says.

His case rests on three mechanisms. The first is recurrence. A process that has been automated keeps returning its saving every month it runs, so the value of a deployment accrues over its working life rather than on the day it goes live, and successive deployments add to one another. The second is capability. Each project leaves the organization with better-documented processes, tested tooling and a team that has been through the change once, which reduces the cost and duration of the next one. The third is reinvestment. Hours recovered from repetitive work can be redirected to revenue-generating activity, so part of the return appears as new income rather than lower cost.

The board-level implication runs against the way AI is usually sold. Hype frames AI as a single transformational bet, and a board that accepts that framing tends to look for one large return and to judge every smaller one as a disappointment. The portfolio Janes describes is a sequence of measurable deployments, each small enough to verify, building a cumulative return that no single project would show. It is also consistent with MIT's finding that back-office processes, where repetitive manual work is concentrated, have produced the strongest returns.

A modest result, on the record

The most recent example Janes cites ran at Reformat Studio, a web agency in his group, which used AI to migrate client websites from WordPress onto a custom architecture. The migrations had been repetitive, slow and disliked by the team, and the before-and-after cost of each one was easy to measure. The deployment took between two and four weeks.

The project saved more than £30,000 in labor over six months, largely by removing the routine work of applying WordPress and third-party plugin updates. The faster migrations also supported a £39-a-month care plan subscription, which generated £10,300 in new annual recurring revenue.

"It's not the multi-millions touted by influencers, but it's still a return, and it will be stacked on top of previous and future projects," he says.

The figures are the company's own and describe one process at one business. Janes presents them on that basis: a single measured result, published with its method, timescale and scope, of the kind he says the market's more extraordinary claims almost never include.