Spend Without Signal: The Measurement Gap Behind Every Runaway AI Budget

Why tracking AI costs isn’t enough – and how connecting usage, adoption, and outcomes turns AI spending into measurable business value.

The Wall Street Journal recently ran a piece with a headline that stopped me mid-scroll: “The Metric CFOs Struggle to Track: AI Usage.”

The data point buried in the second paragraph is the one that should concern every business owner: only 26% of companies say they have a comprehensive view of their AI costs. Half have “some visibility.” And 22% don’t know what they’re spending until the bill shows up.

That’s three out of four companies flying without instruments.

I’ve spent years building automated systems for businesses. That 74% number doesn’t surprise me. It does, however, frustrate me – because it’s preventable.

The bill shock problem is a symptom. The measurement problem is the disease.

The Journal article frames this primarily as a CFO challenge, and fair enough – finance is where it gets noticed. A company blows through its annual AI budget in a few months. Another sees token usage explode sixfold. A third rolls out twelve AI tools to its marketing team, watches usage drop off after a few weeks, and only realizes after the fact that employees drifted back to their previous workflows.

Each of those stories is really about the same thing: AI investment that wasn’t connected to a feedback loop. Spend without signal.

Imagine hiring a contractor, giving them an open tab at the hardware store, and agreeing upfront to not get progress updates and never visit the job site to see how things are going. You see the spend, but you can’t see what it’s buying; whether the work is on track or on spec is unknown until the final invoice lands. No reasonable homeowner would accept those terms with a human crew. But that’s essentially the arrangement most companies have with their AI systems.

The technology runs. Tokens get consumed. And somewhere upstream, someone is waiting to find out if any of it was worth it.

Usage visibility and ROI visibility are two different metrics. You need both.

Knowing how many tokens you spent last month is useful. Knowing whether those tokens moved a business outcome is what turns AI spend into AI investment.

The CFOs in the WSJ piece are grappling with a usage problem – metered pricing, unpredictable consumption, no real-time monitor. That’s real and it needs solving.

But there’s a more expensive problem beneath it: organizations that are spending on AI they can’t prove is working. When Reckitt’s CFO noted that some AI tools saw usage “drop off after a few weeks as employees reverted to old habits,” that’s not a token problem. That’s an adoption problem. A behavior change problem. A change management problem. And without measurement infrastructure underneath those tools, you’d never catch it early enough to course-correct.

The Amer Sports CFO put it plainly – he didn’t want people “setting up AI agents or AI processes that didn’t have long-term scalable benefits.” That’s exactly the right instinct. The question is: how do you know, in advance and in real time, whether the benefit is there?

The Fix Lives in How You Build, Not How You Budget.

There’s a tendency, when costs get unpredictable, to do what Corning did – limit the number of AI tools employees can access and narrow the portfolio to a few major projects. That’s a rational defensive move. But it’s essentially choosing reduced risk over captured value.

The better path is to build AI-powered processes where observability is a first-class requirement – not something you try to bolt on after the fact when the bill gets scary.

This means knowing what each automated workflow is doing at every step. How many tasks completed. How many exceptions routed to a human. How long it took. What it cost per transaction – including the labor and technology components together, not separately. And whether the downstream outcome (the invoice processed, the document reviewed, the customer issue resolved) actually happened correctly.

When you have that, you’re in a position to answer the question every CFO should be asking: did we make money on this?

“Did we spend what we budgeted” is a different, much easier question. Did the investment generate a return that justifies the continued spend? And can we see that in something closer to real time than a quarterly review?

The Edge Will Go to the Companies That Can See Clearly.

That means treating agent observability as infrastructure, not afterthought. It means connecting AI activity to business outcomes before you scale, not after. And it means being willing to look at what the data shows, even when it’s uncomfortable – the way Reckitt was willing to pump the brakes on a solution that was generating inaccurate data.

The CFOs who figure this out will walk into the board meeting with answers at the ready before anyone asks the questions: a cost-per-transaction number, a productivity delta, and an ROI model they can defend – not a billing mystery to explain.

That’s what operational intelligence looks like. And it’s available to any organization willing to build it in from the start.

If you’re not sure whether your AI investments are instrumented for visibility and ROI, I’m happy to take a look. Book a free consultation to start the conversation.

#AIStrategy #OperationalExcellence #AIInvestment #DigitalTransformation #BusinessLeadership #AIGovernance