You’ve just approved an AI investment for your company because you know that upskilling your work force holds the key to greater efficiency and record-pace innovation. To calculate the cost of the new copilot, you have multiplied the monthly subscription fee by the number of users. That seems like a straightforward way to forecast the cost. However, after the rollout, some users adopt the technology quickly, ask to turn on more advanced capabilities, and these newly empowered citizen developers and vibe coders create their own agents and automations. Each agent can call multiple APIs and other agents, and each agent that is called can in turn call more APIs. That consumption leads to budget overruns and it’s how a company can get AI sticker shock.

A recent McKinsey study found that transitioning from isolated use cases to enterprise production raises the cost of AI by an average of 4X. Of the organizations in the survey that have moved beyond piloting use cases, 93% say they have exceeded their AI budget.

AI costs are a concern for companies of all sizes: Uber blew through its annual AI budget in just four months.

To understand why AI deployments can cause sticker shock, it’s important to dig into token costs. Tokens are the fundamental currency of Large Language Models (LLMs). One token equals approximately four characters or about 0.75 words.

Unlike traditional software that requires a flat monthly fee for unlimited clicks, consumption-based models of AI charge you for every piece of information that enters and leaves the system.

Most commercial AI vendors use asymmetric pricing. That means that they charge more (up to four times more) for output tokens than input tokens. For clarity:

• Input Tokens (Prompts + Context): Everything you feed into the AI, including background files, system instructions, and chat history.
• Output Tokens (Completions): The text, code, or answer the AI generates.

Sticker shock because of overages occurs mainly because of chat history and Retrieval Augmented Generation (RAG). When you have a long conversation with AI, the system doesn’t just look at your last input. To remember what you said earlier and what it did earlier, it sends the entire chat history back to the model as input tokens with every prompt. Here’s an example of how that could go:

• Prompt 1: 50 tokens input → 100 tokens output. (Total billed: 150 tokens)
• Prompt 2: 150 tokens history + 50 tokens new input → 100 tokens output. (Total billed: 300 tokens)

If the conversation had 10 prompts, even a 5-word prompt at the end might pull thousands of background input tokens into your bill.

Luckily, there are strategies that can help. One is Model Routing. There are multiple ways to achieve this depending on whether you are using a closed-system or an open-source development stack, but the concept is the same: If a user asks your intelligent agent a simple question like "What are your store hours?", the router intercepts it and sends it to a small model (like GPT-4o-mini) where input tokens cost $0.15 per million instead of $5.00 for the larger, more powerful model. By default, high-token context loads get routed to the cheap model. The expensive model only gets called in when the task actually needs deep reasoning.

Besides Model Routing, FinOps is a tried-and-true best practice that aggregates, presents, and optimizes cloud and AI spend across multiple vendors and all of your users. Through attribution, your AI/Cloud Center of Excellence or FinOps team has the data needed to fulfill showback and chargeback programs for not only the cloud and compute infrastructure used to host your AI platforms, but also data engineering labor, third-party SaaS subscriptions, and proprietary vendor contracts. Instead of viewing a $50,000 monthly AI bill, Finance teams using a product like Apptio Cloudability see exactly how much is being spent on HR’s employee chatbot vs. Engineering’s CoPilot licenses.

Token tracking itself has been difficult because API data from third-party AI vendors isn’t formatted like a standard SaaS cloud bill. To overcome this challenge, the Apptio platform uses the FOCUS (FinOps Open Cost and Usage Specification) standard to ingest and standardize the token data. Once that data is made ready for analysis, a company can visualize the volume of prompt tokens entering the system versus completion tokens exiting it, exposing costly context-window bloating. Through model profiling, token spend is classified across models, instantly highlighting if a team is wasting money by using an expensive model for basic tasks such as text formatting.

Model routing controls what a request costs before it happens. FinOps tells you what it actually cost after the fact, and where the money went. Use both of them together and that $50,000 line item is no longer a mystery. It becomes an item you can act on.

If you are ready to stop overpaying for tokens, you don't have to do it alone. Please Contact LRS to book a meeting with me and our AI specialists, and let’s get you started aligning token spend with business value right from the start.