Why AI commitments are hard to judge from inside
The pressure to be seen doing something with AI is real, and it distorts use-case selection: initiatives get chosen for their visibility to the board rather than their economics, and vendor claims are accepted because nobody internally can technically contest them. At the same time, the sceptics are equally uncalibrated: dismissals based on one bad demo are as common as commitments based on one good one. The result is that AI decisions are frequently made at the extremes of enthusiasm or dismissal, when the actual question is narrow and answerable: for this specific process, at this error rate, with our data, does the economics work?
The questions that sort real use cases from showpieces
- What does the process cost today, per transaction or per decision, measured rather than estimated, and what error rate does the current human process actually run at?
- What happens when the system is wrong: who catches it, how quickly, and what does one uncaught error cost in money, regulation or trust?
- Does the data this depends on exist in usable form now, or does the business case silently include a data programme nobody has scoped?
- Where exactly does the human sit in the new workflow (reviewing everything, sampling, or handling exceptions), and does the maths still work at that level of oversight?
- What does this cost at production volume, including inference, integration and monitoring, as opposed to what the pilot cost?
- If the vendor's accuracy claim is off by ten points on our data, does the case survive?
Blind spots specific to this wave of adoption
Three misjudgements recur. First, comparing AI to perfection rather than to the current process: human-run processes have error rates too, usually unmeasured, and the honest comparison is between two imperfect systems. Second, treating the pilot as evidence of production readiness, when pilots run on curated inputs, tolerate manual workarounds and are attended by the vendor's best engineers; production runs on real data, at volume, unattended. Third, ignoring accumulation: each individually sensible tool adds a dependency, a data-sharing agreement and a monitoring obligation, and an organisation that adopts twenty tools has acquired a governance workload it never explicitly decided to take on.
What to nail down before the spend becomes structural
Pressure-test the accountability chain first: a named owner for the system's outputs, error thresholds that trigger review, and an agreed answer to what we tell the customer or the regulator when a decision was machine-made. Then test reversibility: can this use case be staged so the organisation learns at small scale, or does the vendor contract and integration design mean you are committed before you have evidence? Then test the workforce assumption: if the case rests on people moving to higher-value work, name the work, because savings that depend on unspecified redeployment have a habit of becoming neither savings nor redeployment. This level of uncertainty often warrants independent challenge before the programme is presented for funding.
What experienced operators bring to an AI decision
The most useful outside perspective on an AI commitment is usually not deep technical review but pattern recognition from executives who have run automation and system programmes through the same organisational physics. They have watched accuracy claims survive the lab and die on live data, watched middle management route around tools that threatened their judgement, and watched cost cases evaporate when the exception-handling headcount was finally counted. Their challenge tends to move the decision from whether to adopt AI to which single process to industrialise first. That narrowing, made before budget is committed, is usually the difference between a production system and another pilot in the portfolio.
