“The AI got it wrong” is usually a process problem

A complaint that reaches us regularly: we tried this, it produced something wrong, we stopped. In almost every case we have looked at, the model was not the part that failed.

Four causes, roughly in order of how often they turn out to be the real one.

One: it was never told what « right » looks like

Someone asked for « a summary of this » and got a summary that was not the one they had in their head. There is no version of this where the machine is at fault; the requirement was never stated.

The 23 July post on reusable prompts is the fix, and it takes twenty minutes once.

Two: it did not have the information

The answer was wrong because the correct answer was in a document the tool could not see. People routinely assume that a system connected to one folder can see everything in the company.

Before concluding a tool is inadequate, check what it was actually given. This is the most common cause of « it invented something ».

Three: nobody checked, because nobody was asked to

The output was wrong and went out anyway, because the process had a step for producing and no step for reviewing. That is a process design fault that predates AI by decades — the same thing happened with templates and with mail merge.

Four: the task was a bad candidate

Sometimes the honest answer is that the process varies too much, or the cost of an error is too high to run it unattended. The three questions from 12 September catch this before you build, which is considerably cheaper than catching it after.

Why this matters more than it sounds

Because « we tried AI and it did not work » becomes company folklore very quickly, and it is usually a description of one badly-set-up attempt rather than of the technology.

When you hear it, ask which of the four it was. In our experience you get an answer in five minutes, and about three times in four it is the first or the second.