AI made the mistake. Why am I paying for it?

My LinkedIn feed has picked up a fairly consistent complaint over the last few days. Someone asks an AI to review something, wait for approval or stop. It acknowledges the instruction, then carries on anyway.

I've had versions of it myself. You say, "Don't change anything yet." Something changes. You ask for a narrow bit of research and 20 tool calls later... it has wandered into work nobody asked for. Then you spend longer checking, reverting or explaining the boundary again than the original task should have taken.

AI making mistakes isn't surprising. Paying for the mistake is the part that bothers me.

If the agent ignores a clear instruction and that failure burns paid usage, why should the customer automatically carry all of the cost?

We're very good at metering AI. Tokens, credits, requests and tool calls can all be counted in extraordinary detail. I'd like the same level of attention on what happened when some of that usage came from a failure we can actually prove.

Some failures already aren't charged

There is already some protection when a service fails in a way the provider can clearly identify.

Anthropic's current API billing guidance says customers are billed for successful API calls and completed tasks, while failed requests aren't charged. Its usage-limit guidance also lists tool use, conversation length, model choice and effort among the things that can affect usage.

Google's current Gemini API billing guidance says requests that fail with a 400 or 500 error aren't charged for the tokens used. It also says prepaid credits are generally non-refundable and that long-running work, including agents, can continue consuming credits briefly because the billing system cannot stop usage instantly.

OpenAI's current Services Agreement says usage-based services are charged according to usage calculated by OpenAI and that fees are generally non-refundable except where the agreement or law says otherwise. Its Service Credit Terms say purchased service credits are generally non-refundable, while the current guidance for personal flexible-usage credits says credits may be restored where confirmed account compromise caused unauthorised use.

Those are sensible protections. They deal with failures the service can identify fairly cleanly: a request errors, the service fails, or somebody uses an account without permission.

The gap I'm interested in is when the request technically works. The model answers, the tools run and the usage record looks valid, but the system still did the wrong work.

A green tick can still mean the job went wrong

If you tell an agent to review a repository without changing anything and it changes files, the request may still look technically successful. If you ask it to wait before sending something and it sends it anyway, the email service probably worked perfectly. If you give it a narrow research task and it heads down several side roads, every search and model call can still complete normally.

From the billing system's point of view, that can be a lovely row of green ticks. The model ran. The tools ran. The tokens were counted.

The person using it is left asking, "Why on earth did you do that?" and then has to check, undo or repeat the work.

Agents make this more expensive than a bad chat answer because they can create files, call services, search systems, run code and use paid third-party APIs before you see what happened. A poor decision can trigger more work very quickly.

The customer cares whether the job was done properly. A technically successful API call only tells part of that story.

The usage bill only shows the easy number

The usage bill is precise, while the person's recovery time is much harder to see. A 5-minute task becomes 40 minutes of checking and correcting. Irrelevant work fills the context and can make the next attempt more expensive. Tool calls can create charges somewhere else. A coding agent can leave changes that another person now has to review or revert.

After enough of that, people start watching the AI more closely. We give agents more freedom because we want to save time, then end up sitting beside them like nervous driving instructors.

That doesn't mean the provider should pay every time something goes badly. Cost per token just doesn't tell us whether the money achieved anything useful.

The FinOps Foundation's guidance on unit economics already describes AI measures moving beyond cost per token towards things such as cost per assist, cost per agent action and cost per case deflected. Those measures are much closer to how a business thinks about value.

I'd add a simpler question beside them: did the AI do the work we actually asked it to do?

Keep the result beside the meter

For any AI run that matters, I want to be able to see something like this:

what we asked for
  -> what the agent decided to do
  -> models and tools it used
  -> what it cost
  -> what happened
  -> why it failed, if it failed
  -> what usage should be charged

OpenTelemetry's current GenAI observability work describes traces that can include model calls, tool calls, prompts, responses and token use. The raw evidence needed to understand an agent run is becoming much easier to collect.

Beside that, I want a plain record of whether the job succeeded and whether the agent stayed inside the instructions it was given.

That gives us something better to compare with the bill: what did we agree the job was, did it do it, and if not, why?

Not every bad answer deserves a refund

I don't want a refund button attached to every thumbs down. AI can misunderstand vague instructions, difficult work can genuinely need several attempts, and people can disagree about whether an answer was good enough.

My inner lawyer wakes up a bit when the instruction and the trace are both clear.

If I say, "Don't run anything until I approve it", and the trace shows that the agent ran tools anyway, we know what happened. If it acknowledges a stop instruction and carries on making calls, we can see that too. If a provider error causes the same operation to repeat several times, that should be visible in the run history.

That's evidence, not taste.

In those cases I don't think the only answer should be, "You used the tokens, so that's the bill."

Cash refunds for tiny amounts would be silly. The provider could return some allowance, issue credits or leave clearly failed usage out of the charge. I don't really mind which method is used. If the failure is clear, the customer should get something back.

Cloud services already work this way in a limited sense. Google Cloud's Cloud Run SLA, for example, provides financial credits when measurable availability targets are missed and the customer meets the claim requirements.

Uptime is much easier to measure than whether an AI agent followed a task properly, so the comparison only gets us so far. It does show that crediting a customer when a provider can measure a failure clearly isn't a particularly strange idea.

The same provider can supply the agent and sell the usage

I'm not suggesting providers deliberately make agents waste tokens to make more money. I haven't seen evidence for that, and it would be a lazy accusation to make without it.

If an agent uses 100,000 tokens and 20 tool calls on something that should have been far smaller, most of the immediate cost sits with the customer. If a clearly failed or unauthorised run could lead to some of that usage being credited back, the provider has a stronger reason to improve stopping, instruction following and tool use, while the customer has a reason to write clearer jobs and boundaries.

The customer has responsibilities too. If you give an agent a vague brief, no useful limits and no approval points, then complain that it explored too much, that isn't obviously a provider failure.

A clear job, clear boundaries and clear approval points give everyone something real to compare the run against.

A budget can't tell you whether the job was done

We already have good tools for setting budgets, limiting spend and watching token use. They tell us how much the AI is allowed to spend and how much it actually spent.

They don't tell us whether it did the job.

For any agent doing material work, I'd want to know what we asked it to do, what it was allowed to do, what it actually did, what it cost, what result came back and who owns the next decision.

Once those things are recorded, the bill has some context. Without them, all we really have is a usage total and an argument about whether the result was worth the money.

Most of the work happens before the run starts

Everything above is about proving what happened after the fact. There's an earlier version of the same problem, and it matters more: getting the instructions, the permissions and the approval points right before an agent goes near real work, so there's less left to prove afterwards.

Setting an agent up properly rarely starts with the model at all. It starts with deciding what it can touch without asking, what needs a person to say yes first, and what "stop" is meant to trigger when someone types it. A vague brief with no boundaries will wander, whichever model sits behind it. A brief with a narrow scope, clear approval points and a plain definition of "done" gives the agent something to be held to, and gives the person running it something solid to check the result against.

That's the part that sits alongside everything I've argued for above. The trace and the outcome table matter because they tell you what happened. The scoping, the permissions and the approval gates are what make "ignored an instruction" a rare finding rather than a weekly one.

We can measure more than the bill

Providers know how many tokens were used. Agent systems can record the calls and tools. The people who set the agent up in the first place know what boundaries it was given and what it was actually asked to do. Businesses know what result they were trying to get. Put those together and we can see what happened without pretending every disappointing answer is a provider failure.

I don't expect a refund for every hallucination, weak paragraph or recommendation I don't like. I do expect systems to get better at spotting a run that ignored a clear instruction, acted without approval or failed in a way the trace can prove.

If I said wait, the system acknowledged wait and then ran the tools anyway, I shouldn't need to spend another 30 minutes persuading support that the first 30 minutes were wasted.

We have become very good at measuring what AI consumes. We need to get better at measuring whether it should have consumed it.

This is the part of it I actually spend time on with clients: getting the boundaries, the permissions and the approval points set up properly before an agent goes anywhere near real work, then building the record that proves whether it stayed inside them. If that's where you're stuck, it's the kind of thing Jawwws helps with.