On OpenRouter, AI agent traffic is now using about five times as many tokens as human-driven traffic. According to data highlighted on August 21 by the U.S. venture capital firm Andreessen Horowitz (a16z), agentic usage has grown roughly 14-fold since February, and more than 85% of it consists of cached input.

Daniel Newman, CEO of Futurum Group, also suggested in a September 30 post on X that this ratio could widen to 10x or more.

But five times the tokens does not mean five times the inference cost or the bill. Cached input, which reuses the same context, is priced differently from fresh input and output. And as more agents repeatedly handle large amounts of context, the need extends beyond compute to the systems that hold and reuse that information.

To think about demand growth from AI agents, it helps to separate the volume of tokens processed, the amount actually billed, and the memory and storage that support reused context.

AD

What the "roughly 5x" figure on OpenRouter actually means

The chart published by a16z classifies token usage on OpenRouter as "agentic," "human," or "mixed," and shows it as a seven-day moving average.

As of August, agentic usage stood at 7.3 trillion tokens and human usage at about 1.4 trillion, a gap of roughly five times.

The key point is that this number is not a comparison of one AI agent and one person doing the same job.

It compares the tokens processed across all of OpenRouter by API usage classified as agentic with the tokens from usage classified as human. It does not directly show the number of users or agents, or how efficiently a single task gets finished.

Also, the 7.3 trillion figure is not a total for the past seven days; it is a usage level expressed as a seven-day moving average.

According to Peter Walker of OpenRouter, who explained the data, agentic token volume overtook human volume on February 6, 2026. Over the following six months or so, agentic usage grew 14-fold. Human usage also grew 2.8-fold over the same period.

In other words, human use of AI did not decline. What grew even faster was a pattern in which a person sets a goal and software keeps working by calling models over and over.

The classification method also deserves attention.

According to Walker's explanation and OpenRouter's official write-up, seven indicators, including tool-call frequency, number of conversation turns, and response intervals, are combined to classify each API key as agentic, mixed, or human.

Individual requests are not labeled one by one as "human" or "agent." If one API key is used for multiple purposes, the whole key is classified by its dominant behavior pattern.

So the "5x" figure does not mean five agents are running per user, nor that a single task needs five times the compute. What it shows is that the makeup of tokens flowing through OpenRouter is tilting rapidly toward agentic use.

"85%+" and "about 70%" rest on different aggregation conditions

a16z says more than 85% of agentic token usage is cached prompts. Walker, meanwhile, says that for the average agentic request, about 70% is cached.

The public information is not enough to fully reconcile how the two were calculated.

Public statement Cached share What the figure refers to
a16z article, August 21 85%+ Agentic token usage observed on OpenRouter
Peter Walker's explanation About 70% Total tokens in the average agentic request

Sources: a16z article, Walker's post.

When the share is calculated across all tokens versus averaged per request, a few large requests with long contexts can carry different weight in the result. However, it has not been confirmed that this explains the gap between 85% and 70%.

Rather than treating one of the numbers as wrong, it is safer to read them as figures with different aggregation conditions.

What the two have in common is that agents refer back to past context at a very high rate.

Take an agent that fixes code. It keeps the initial task instructions, repository information, tool definitions, and the changes made so far, while reading new test results and deciding on its next step.

The more turns there are, the more the earlier context, not just newly added information, makes up a large part of the input.

Still, "cache" here does not mean a system that simply returns a previous answer.

OpenRouter also has a separate "Response Caching" that returns an earlier response as is for an identical request, but the focus of these statistics is Prompt Caching.

With Prompt Caching, the parts of the input that are the same as before, such as the system prompt, tool definitions, and past conversation, are reused, and the model generates its next output together with the newly added information.

For that reason, a high number of cached tokens alone does not show that an agent is pointlessly repeating the same mistakes. The chart does not include task success rates or the quality of deliverables.

AD

A 14x rise in tokens does not necessarily mean a 14x bill

According to OpenRouter's explanation of Prompt Caching pricing, published July 21, the unit price for input tokens read from cache is 0.1 to 0.5 times the regular input price, depending on the provider.

If the long system prompts, tool definitions, JSON schemas, and work history that an agent sends every time can be reused, it costs less than processing the same amount of input from scratch each time.

So with the same model and pricing terms, the higher the cached-token share, the more likely it is that total token volume and billed amounts will grow at different rates.

However, the savings apply only to the portion that can be read from cache.

Newly added input and the output the model generates are charged at regular rates. Some providers also price the initial "write" that creates the cache at more than regular input.

For example, in OpenRouter's explanation, for Anthropic a cache write held for five minutes costs 1.25 times regular input, and one held for an hour costs 2 times. If a cache is created once and never reused, it can end up costing more than regular input.

Therefore, a16z's "more than 85% cached" cannot simply be read as "costs can be cut by 85%."

There are also conditions for the cache to work.

It cannot be reused if the retention period expires or if the beginning of the cacheable prompt changes. And with a service like OpenRouter that routes requests across multiple inference providers, a request sent to a different provider than last time may find no usable cache there.

To reduce this, OpenRouter uses "Sticky Routing," which preferentially sends requests that follow a Prompt Caching hit back to a provider holding the same cache.

Developers can also specify a session_id to make it more likely that a session is sent to the same provider from the start.

However, if that provider becomes unavailable, requests fall back to another one, so the cache is not guaranteed to be usable for every request.

To manage an AI agent budget, it helps to look not only at total tokens but also at

  • new input
  • input read from cache
  • cache writes
  • output

separately.

That makes it easier to distinguish why token volume rose from why actual spending rose.

Prompt Caching and the KV cache are not the same number

The large volume of cached tokens is also related to memory demand in inference infrastructure. But there is an important distinction here.

The "cached tokens" shown in OpenRouter's statistics count input tokens processed using the cache, from an API and pricing perspective.

When an LLM actually runs, by contrast, it uses a KV cache, which stores the Keys and Values obtained in attention computation to avoid recalculating them.

With Prompt Caching, a provider may be able to reduce processing by reusing such computed states. The specific storage method, however, differs by model and provider implementation.

Therefore, the fact that 85% of tokens on OpenRouter were cached does not mean the information corresponding to that 85% is sitting as is in GPU HBM.

Nor can the required HBM capacity be calculated directly from a cumulative processing volume like 7.3 trillion tokens.

If the same context is referenced many times, the number of tokens processed as cached rises. But if the same information is being reused, the data actually stored does not necessarily keep growing at the same rate.

The amount of memory needed varies with factors such as

  • the number of agents or sessions running at the same time
  • the context length of each session
  • the model's number of layers and attention structure
  • the data type of the KV cache
  • mechanisms for sharing and compressing the cache
  • how long it is retained

AD

Long contexts that HBM alone cannot support

As agentic processing runs for long periods and holds large amounts of context simultaneously, it becomes difficult to keep everything in the GPU's high-speed memory alone.

In a technical explanation published March 16, NVIDIA describes an approach in which context such as the KV cache is placed across multiple tiers according to usage frequency and required speed.

Storage tier Main role described by NVIDIA
GPU HBM (G1) The fastest KV cache, used directly during token generation
System DRAM (G2) Staging and buffering for KV cache offloaded from HBM
Local SSD (G3) "Warm" KV cache reused after a short interval
Shared storage (G4) Durable data such as history and artifacts, removed from immediate inference processing

The closer to the GPU, the faster the access, but the more limited the capacity. Moving to more distant DRAM, SSD, or shared storage increases capacity, but the transfer time to bring data back to the GPU becomes an issue.

NVIDIA's CMX adds a network-attached flash tier called "G3.5" between G3 and G4.

The aim is to hold large amounts of KV cache while transferring information that will be needed again to the GPU or host memory in advance, reducing the time generation stalls waiting for data.

The important point is that simply adding capacity is not enough.

If the context an agent needs cannot reach the GPU by the time it is next used, inference has to wait even when capacity is available. The growth of agents makes not only compute but also the ability to store context and move it where it is needed a factor in infrastructure performance.

However, this is a design NVIDIA presents for its own AI infrastructure, and it does not mean OpenRouter stores its cache in the same configuration.

OpenRouter's usage statistics cannot be directly tied to NVIDIA's infrastructure design to estimate the HBM or SSD capacity required.

Even so, it shows why the growth of agentic AI cannot be seen as demand for GPU compute alone. Long-running processes need not just the ability to compute but also mechanisms to reuse past context efficiently.

Even if "5x" becomes "10x," value will not necessarily grow 10x

Daniel Newman suggests that the gap between agentic and human usage, now about 5x, could widen to 10x and beyond.

This differs from the current ratio observed by OpenRouter; it is a forecast about the future. Because no timeline or calculation model was given, it cannot be read as "10x is certain."

What the OpenRouter data shows is that it is becoming harder to measure corporate AI demand simply by "how many people use AI."

If a person gives a model one instruction and, behind the scenes, an agent searches, writes code, runs tools, reads results, fixes problems, and calls the model again, a large amount of inference occurs with almost no increase in the number of interactions with the person.

Even with the same 100 people using AI, the compute required differs greatly between those who only chat and those who run multiple agents for long periods.

On the other hand, the sheer number of tokens consumed cannot tell you whether the investment created value.

An agent that completed a job with 100,000 tokens and one that used 1 million tokens and failed midway cannot be judged as the latter representing greater demand or value.

What to measure is the cost per completed task

As the use of agentic AI grows, the metrics companies should watch change as well.

Beyond the simple price per token or the monthly cost per user, it is necessary to look at how much it ultimately cost to complete one task successfully.

For example, measure together:

  • total tokens until the task is completed
  • actual inference cost after caching
  • time to completion
  • success rate
  • time required for human review and correction
  • cost spent on failures and retries

Even if caching lets large amounts of context be reused cheaply, it is not necessarily efficient if the agent fails repeatedly and people have to fix its work. Conversely, even with large token volumes, it may be worth the cost if a task that took a person hours is reliably finished in a short time.

The fact that agentic token volume on OpenRouter has reached about five times human volume shows that the center of AI demand can no longer be captured by simple "human and AI conversations" alone.

The longer agents run autonomously, the more costs are shaped not only by model compute but also by cache pricing design, the memory that holds context, and the mechanisms for storing and moving it.

If this scale expands further, what matters will be not just "how many tokens were used." It will be how many tasks those tokens actually completed.