Measuring AI ROI before you've spent the money

Most AI projects don’t fail because the technology didn’t work. They fail because nobody agreed what success looked like, so after three months the team has a working pilot and no way to say whether it was worth the money. Leadership asks “what did we get for this?” and the answer is a shrug.

The fix is to choose the number before you start, not after. It sounds obvious, and it’s the step most teams skip.

Pick a measure that counts something real

“See if AI helps with invoices” is not a measure. “Cut the time to process a supplier invoice from 12 minutes to 4, with no increase in errors” is. The difference is that the second one gives you a clear yes-or-no at the end of the pilot.

A good measure has three properties. It’s countable — hours, error rates, turnaround time, not vibes. It’s specific — one task, one number. And it has a baseline — you know what the number is today, before the AI touches anything.

The baseline is the part people forget. If you don’t measure the current state first, you have nothing to compare against, and every claim afterwards is guesswork.

Cost the whole thing, not just the model

The model is rarely the expensive part of an AI project. The real cost is the integration work: connecting to systems, cleaning data, building the workflow around the model, and the ongoing monitoring. If your ROI calculation only counts the API bill, you’ll be surprised later.

Cost both the build and the run. Build is the one-time effort to get it live. Run is the recurring cost — model usage, compute, the person who reviews the outputs, the time spent fixing errors. Run cost is the one that quietly erodes an ROI that looked great on paper.

Track the honest version of the number

The most useful AI metric isn’t the optimistic one. Track three numbers per task, and be honest about all three:

Time per task. Does the work actually get faster, measured against the baseline?

Error or rework rate. If the AI is faster but introduces mistakes that someone else has to fix, the saving may be an illusion.

Adoption. How many of the people who could use it actually do? A tool that saves an hour but only two people use it isn’t moving the business.

Most teams track the first and ignore the other two. The projects that survive are the ones that watched all three.

Decide what you’ll do if it misses

Agree in advance what happens if the number falls short. A small project that fails cheaply and teaches you something is a useful result — but only if you write down the lesson. “We tried this, the data wasn’t clean enough, here’s what we’d need before retrying” is an outcome leadership can act on. “We tried this” is not.

Where this fits

This is the same discipline we use in our AI readiness assessments, where every pilot plan includes a written measure of success before work starts. If you’re still choosing which project to measure, start with Choosing your first AI use case, then come back here to put a number on it.