The recommendation nobody ordered
The pricing model, the network optimisation, the capital-allocation tool: after months of investment it finally produces a clear recommendation, and it is not the one the executive team expected. There is a pause. Then someone asks the question that defines this pattern: can we look at the weightings? Everyone in the room understands what is really being asked, which is whether the model can be adjusted until it agrees.
Why the divergence keeps happening
The model optimises the objective it was given, exactly and narrowly. The humans in the room are carrying constraints nobody encoded: a relationship with a key account, a board sensitivity, a regulatory mood, a memory of the last time something similar failed. Both parties are working with partial information, and neither can see the other's. The disagreement persists because both escape routes feel safe. Overriding the model carries little personal risk, since no executive has yet been dismissed for preferring their own judgement, and deferring to it feels safe too, because responsibility appears to transfer to the machine. Both routes avoid the actual work, which is finding out why the two judgements diverge.
Four ways it goes wrong
- Weightings are adjusted until the model agrees with the room, at which point the organisation has bought an expensive mirror and called it analysis.
- The room defers wholesale, including in situations far outside the conditions the model was built on, because "the system recommended it" is a comfortable place to stand.
- The recommendation is dismissed wholesale, and the signal it carried, often an uncomfortable truth about a favoured option, is discarded with it.
- No record is kept of when the model was overridden and what happened next, so the organisation never learns whether its judgement or its model performs better.
Treat the disagreement as a diagnostic
The divergence is information before it is a problem. Locate its source: do the model and the room hold different inputs, different objectives, or different constraints? Each has a different remedy. Missing inputs can be added; a mis-specified objective is a leadership conversation, not a data one; an unencoded constraint should be named aloud and examined, because some constraints turn out to be preferences wearing a suit. Overrides should be permitted, explicit and owned: a named person, a written reason, and a log that is revisited. And never retune the model in the middle of the decision it is disagreeing with. Change the parameters between decisions, on evidence, or the tuning becomes a way of laundering preference through software.
The judgement layer between model and room
The hardest question in this pattern is not technical: it is whether this particular disagreement is the model surfacing something the room cannot see, or the room knowing something the model cannot. Senior operators who have deployed and overridden decision models in production carry exactly that pattern recognition. Independent perspectives from people with no stake in either the tool's credibility or the room's preference can say which kind of divergence this looks like, before the organisation commits in either direction.