AI is becoming part of everyday enterprise work. According to Deloitte’s 2026 State of AI in the Enterprise, companies expanded employee access to AI by 50% in 2025, while the share of employees with access to approved AI tools rose from less than 40% to about 60%. The same report found that 85% of surveyed organizations expect employees to customize AI agents for their own work. Together, these findings show how AI is becoming a broader layer of work infrastructure across the enterprise.
As AI usage expands, the operating question becomes more concrete. In an environment where thousands or hundreds of thousands of employees use AI every day, even small LLM calls from individual users accumulate into organization-wide operating costs. The pressure becomes more visible when AI agents call multiple tools, repeatedly reference context, and process similar requests at scale. Enterprises need a way to forecast and control those costs before they become difficult to manage.
Gartner projected in 2026 that AI inference costs for agentic workflows could increase by more than fivefold by 2028. The analysis points to longer reasoning, more tool calls, and larger business context as drivers of higher workflow-level cost.
Large technology organizations are already seeing this issue in their operating metrics. Uber Engineering recently reported that weekly active users of its agentic tools increased 7x and weekly agentic requests increased 9.4x from February to August 2026, while overall AI spend remained relatively stable after April. The case makes the operating challenge clear: as AI usage grows, enterprises need to manage token usage, model requests, cost per session, and tool call structure together.
This is why LLM cost optimization is becoming an enterprise operating priority. A model price sheet captures only one layer of cost for enterprise AI chatbots and AI agents. The final cost depends on how many reasoning steps AI performs for a single user question, how many tools it calls, and how much business context is sent with each call. In enterprise environments, customer information, product structures, order status, contract terms, pricing policies, inventory, and internal exception rules often need to be connected. That is where cost differences become visible.
The Repeated Call Structure Behind Unexpected LLM Cost Growth
When an AI agent handles work across multiple steps, the prior conversation and query results need to be carried into the next step as input. To continue making decisions, the previous conversation, tool call results, reference documents, and decision rationale are sent again. It is similar to a meeting where every person rereads the previous meeting notes before speaking. By the tenth comment, the context from the previous nine comments is sent again.
In this structure, a large document or query result introduced early continues to affect later calls. As calls continue, the same information is counted again toward input cost. A large document, long query result, or detailed business context introduced early is sent again across the remaining calls. The actual cost is shaped by both the size of that information and the number of repeated calls that follow. To the user, it may look like one question and one answer. Inside the system, multiple model calls, tool executions, intermediate interpretations, and context retransmissions may be taking place.
The graph is a simplified way to understand which variables drive cost. In the formula, ‘a’ represents the base input repeated on every call. This includes system prompts, tool descriptions, and user requests that are sent whenever an AI agent starts work. ‘k’ represents the context that accumulates as calls continue. Tool call results, query results, intermediate reasoning, and conversation history become information that can be reflected again in later calls. ‘n’ represents the number of calls required to reach an answer.
Base input grows in a relatively predictable way as the number of calls increases. Accumulated context behaves differently. The longer the call sequence becomes, the more context is included again in subsequent calls. A workflow that reaches an answer after 12 rounds of exploration and verification follows a very different cost curve from one that finds the required information within 4 calls. From the perspective of accumulated context, reducing the number of calls can have a significant impact on total cost.
For this reason, enterprise AI agent cost management needs to consider prompt length, repeated calls, and accumulated context together. Teams need visibility into which workflows repeatedly carry context, which tool calls are duplicated, and which questions lead to long loops. Cost can be controlled when teams can see where it accumulates.
Three Variables That Reduce Token Usage
As shown in the table, the three variables affect cost in different ways. Base input is repeated on every call, so it can be reduced in a stable way when the input structure is cleaned up. Accumulated context grows as call sequences get longer, so teams need standards for preventing query results and intermediate judgments from piling up unnecessarily. Call count affects both input and output costs, which makes it directly tied to how efficiently an AI agent reaches an answer.
LLM cost optimization starts by observing these variables separately. Teams need to identify which costs come from base input, which come from accumulated context, and which come from repeated calls. This visibility helps determine where optimization should begin.
Operating Standards for Repeated Input and Accumulated Context
Managing repeated input starts with reviewing system prompts and tool descriptions. The content sent on every call should be concise and clear. Tool descriptions should be specific enough for the agent to determine when each tool should be used. Clearer tool descriptions can reduce incorrect tool calls, which can also reduce unnecessary calls and cost.
The impact of this work depends on the execution environment. When prompt caching works reliably, the cost burden of repeated base input can be lower. When caching frequently breaks, or when prompts change often based on user permissions and work conditions, repeated input becomes an important cost driver.
Accumulated context needs its own management standards. If a large query result remains in the conversation as is, it is sent again on later calls. When results are too large, teams can keep representative samples, preserve only the latest result in full, and replace older results with short summaries.
Compression needs to be designed with quality standards. If necessary information is removed, the agent may call another tool to verify the missing context. That can increase total cost again. Teams need to measure and compare how much compression is possible while maintaining answer quality.
Ontology Search Shortens the Path to an Answer
Ontology structures the business concepts and relationships distributed across an enterprise. It defines how concepts such as customers, products, orders, contracts, policies, pricing, and inventory connect with each other. It also creates a foundation for determining which data and rules a question should reference.
With this knowledge structure, an AI agent is less likely to reread a broad range of context every time it solves a problem. It can interpret a user question in terms of business concepts and prioritize the data and tools connected to those concepts. Pricing questions can be routed toward pricing policies and product attributes. Inventory questions can start with warehouse and SKU status. Customer inquiries can first reference contract terms and support history.
Ontology search also improves tool call efficiency. When AI searches broadly every time it needs to choose a tool, loops become longer. When the relationship between business concepts and tools is structured, the candidate tools for a given request can be narrowed. The fewer attempts required to find the right information, the lower the call count and the shorter the accumulated context.
Solution Cache Reduces Repeated Exploration
Solution Cache reduces call count for recurring business questions. It works by storing human-validated answers or problem-solving paths, then helping the agent reduce exploration when a similar question appears.
For example, a user may ask, “Show me July revenue,” and then follow up with, “Show me August as well.” For a human, this is almost the same type of request. If AI can refer to the metric definition, data source, and calculation conditions used previously, it does not need to repeat the same exploration from the beginning.
Solution Cache also needs to be evaluated through both cost and quality. Providing a cached solution path as a hint can slightly increase the repeated input included in the prompt. If that hint reduces enough exploration loops, total cost can still decrease. Solution Cache does not need to be injected into every question. Teams should compare the additional input cost from the hint with the reduction in exploration loops, then define the conditions for using the cache. The decision should rely on measured cost savings and quality impact.
Cost Optimization Needs to Be Designed With Quality
The highest-risk approach to LLM cost optimization is reducing call count at the expense of quality. Faster answers can still create larger downstream costs when incorrect responses are used in business decisions. In enterprise environments, inaccurate AI responses can lead to rework, approval delays, customer response errors, incorrect inventory decisions, and pricing policy mistakes.
Enterprises need to manage token usage together with execution quality. They need to know which information was referenced, which tools were called, which intermediate results were retained, and which context was compressed. These standards make it possible to manage cost and quality together.
Enhans uses ontology-based AgentOS to structure enterprise business context and execution conditions. It helps AI agents reference the information they need, call the right tools, and process recurring workflows through shorter execution paths. LLM cost optimization is both a technical task of reducing token usage and an operating task of ensuring that AI agents work according to the organization’s business standards.
For enterprise AI chatbots to move from PoC into real business systems, cost needs to become predictable. Teams need visibility into which workflows create cost, which questions lead to repeated loops, and which context is repeatedly sent again. Reducing repeated calls, cleaning up base input, and structuring repeatable business knowledge are starting points for sustainable enterprise AI operations.
in solving your problems with Enhans!
We'll contact you shortly!