A market report released by Andreessen Horowitz (a16z) on September 30, State of Markets II, shows that AI token prices are falling while NVIDIA H100 rental prices have held up. Data through August 2026 shows that the two prices, which both fell in early summer, have moved in different directions since July. If cheaper AI opens up new uses and drives more demand for compute, that would be a tailwind for NVIDIA, which sells GPUs. But the same report also shows rental prices falling for newer-generation GPUs. To read demand growth from these price movements, we need to understand what each indicator measures and how it relates to actual usage.
Token Prices Keep Falling While H100 Rental Prices Recover
On page 34 of the report, a chart overlays H100 rental prices compiled by Ornn with Silicon Data's token price index. Citadel Securities assembled the chart from Ornn, Silicon Data, and Bloomberg data, and it shows price movements through August 2026. H100 rental prices are plotted in dollars per hour on the left axis, and the token price index is on the right axis.
Between May and June, both prices fell. From July to August, however, token prices fell further while H100 rental prices recovered.
a16z links this movement to the Jevons paradox: the idea that lower AI usage costs can actually increase demand for compute. The report's introductory article also suggests that as AI gets cheaper, demand grows and older-generation GPUs hold their value.
When AI costs less to use, it becomes easier to adopt for tasks that previously did not justify the expense. For example, a process that once generated a single answer can now add searching, verification, and retries.
If the new usage that results exceeds the compute saved through efficiency gains, total compute demand actually rises. The idea is that better GPU performance does not necessarily reduce GPU demand.
However, what this chart directly shows is only the movement of two kinds of prices.
It is not an experiment testing how much AI usage increased because of price cuts, nor is it a statistic tallying total GPU running hours.
Although a16z's interpretation has a certain logic, the fact that the two prices moved differently does not prove the cause.
The "Expenditure Index" Does Not Measure Total AI Market Spending
Silicon Data's indicator is called the "LLM Token Expenditure Index," but it does not represent total spending across the AI market.
According to Silicon Data's official definition, the index shows the usage-weighted effective price per million tokens. A token is the basic unit AI models use when processing text and other inputs.
| Indicator | What it measures | What it cannot tell you on its own |
|---|---|---|
| H100 rental price | The fee to rent a GPU for one hour | Total rental hours, utilization, supply |
| Token price index | The effective price per million tokens, reflecting the mix of models in use | Total token volume, total market spending, individual companies' bills |
The average price can fall for reasons other than price cuts by AI model providers.
Even if the price of each individual model stays the same, the index falls if users shift to cheaper models.
Silicon Data also explains that changes in the mix of models in use and in the ratio of input to output tokens affect the index.
In other words, the price decline in the chart may include both the factor of "the same AI model getting cheaper" and the factor of "cheaper AI models being used more."
Caution is also needed on how GPU rental prices are compiled.
For example, Silicon Data's GPU price index converts various data into a price per GPU per hour and aggregates it after adjusting for differences in contract type, region, and hardware configuration.
The H100 price trend in this report, on the other hand, uses Ornn's data. It would therefore not be appropriate to add H100 prices that Silicon Data publishes for other dates to the same chart for comparison.
The movements of the two prices are an important clue for exploring changes in the AI market.
But a halving of the token price index does not mean AI companies' revenue also halved. Likewise, a rise in H100 rental prices does not mean usage grew at the same rate.
H100 Up 14.7%, B200 Down 5.3%: Prices Move Differently by GPU Generation
In a separate chart, a16z also shows rental price trends by GPU type.
The chart in the lower part of page 37 of the report indexes prices to 100 on August 1, using end-of-August data compiled by Ornn Exchange.
The chart shows that even within the same month of August, prices moved in different directions depending on the GPU type.
The H100 price index rose 14.7%, while B200 fell 5.3% and H200 fell 9.1%.
| GPU | Final index (Aug. 1 = 100) | Change since Aug. 1 |
|---|---|---|
| H100 SXM | 114.7 | +14.7% |
| A100 SXM4 | 96.4 | −3.6% |
| B200 | 94.7 | −5.3% |
| H200 | 90.9 | −9.1% |
The rates of change were calculated by subtracting the baseline of 100 from the final index shown in the chart. For example, H100 is 114.7 − 100 = 14.7%, and B200 is 94.7 − 100 = −5.3%.
All are comparisons over the same period using August 1 as the baseline; they do not compare GPU performance or actual hourly rates.
The 14.7% rise for the H100 certainly stands out.
But not every GPU rose in price over the same period. Prices fell not only for the newer B200 and H200 but also for the A100, which is older than the H100.
The simple view that older products quickly fall out of use once new GPUs arrive has trouble explaining the H100's resilience. Yet the view that growing AI demand lifts prices for all GPUs does not match this data either.
If suitable uses and supply volumes differ by GPU, rental prices may move separately.
However, this chart alone does not reveal how much each factor contributed to each price change.
Nor can the movement of the price index over the single month of August be used to infer global GPU contract prices or future demand.
When AI Gets Cheaper, Does Spending Rise, or GPU Running Time?
When the same type of tokens is used under the same pricing terms, total spending is determined by unit price × usage.
For example, suppose the token unit price is halved. If usage doubles, total spending is unchanged; if usage triples, total spending rises to 1.5 times.
Even if each individual process gets cheaper, total spending grows if usage increases faster than prices fall.
AI agents tend to encourage this kind of growth in usage.
When a person assigns a single task, an AI agent searches for the information it needs, reads the results, verifies its output, and decides on the next step.
In this series of operations, the AI model is called repeatedly without a human issuing instructions each time.
When inference costs fall, such iterative processing can be run at lower cost. However, more token usage does not necessarily mean a proportional increase in work output.
When thinking about GPU demand, improvements in processing efficiency must also be considered.
Assuming the same model is used and conditions such as the input/output mix, quality, and response time do not change, the required GPU time can be roughly expressed as follows.
Required GPU time = tokens processed ÷ tokens one GPU can process per hour
Even if token usage triples, if each GPU's processing capacity quadruples, the required GPU time becomes 3 ÷ 4 = 0.75 times.
This is an assumption to explain the mechanism, not a figure showing actual movements in the AI market.
The key point is that the conditions under which total AI spending grows and those under which GPU running time grows are not necessarily the same.
Processing efficiency is not determined solely by GPU generation or semiconductor performance.
According to NVIDIA's explanation of inference optimization, the stage where input is processed in bulk and the stage where output is generated sequentially use the GPU differently.
Processing capacity also improves by raising GPU utilization through batching, which handles multiple requests together, and by reducing recomputation through caching of past results.
For that reason, a total token volume that mixes different models and cache usage conditions cannot be directly converted into GPU running time or power consumption.
For the view that growing AI usage will also expand demand for NVIDIA GPUs to hold, usage growth must outpace improvements in processing efficiency.
What Matters for NVIDIA Is Whether Efficiency Gains Create New GPU Demand
NVIDIA itself highlights running AI at lower cost as a major feature of its new products.
In its January 5 official announcement of the Rubin platform, NVIDIA said it can cut the cost of inference tokens by up to one-tenth compared with Blackwell.
This is NVIDIA's stated figure, assuming advanced reasoning and large-scale MoE models, and does not mean the API prices users pay will uniformly fall to one-tenth.
It is interesting that NVIDIA, which sells GPUs, is actively pushing to lower the cost of GPU-based processing.
The aim is likely not just to let existing work be handled with less equipment. It is probably to extend AI to uses that previously did not justify the cost and to increase compute demand across the whole market.
If the cost of each process falls and new uses keep emerging, efficiency gains could in fact spur capital investment.
However, not every company involved in the AI market benefits in the same way.
The profits of AI model providers vary with the gap between the selling price per token and the actual processing cost, as well as how they bill users.
Even if token usage grows, revenue and profit will not necessarily rise in step if prices fall faster.
Also, the party that receives H100 rental fees is the operator that owns and leases the GPUs. That income does not directly become NVIDIA's revenue.
A lasting tailwind for NVIDIA would come when existing GPUs alone cannot handle the increased demand and additional capital investment is needed.
Conversely, if improved processing efficiency can absorb the additional demand, growth in AI usage will not necessarily lead to more GPU purchases.
If the cost of using AI is too high relative to the results users obtain, demand growth itself could also slow.
a16z's report offers interesting material for considering the relationship between cheaper AI and GPU demand. But price movements alone cannot establish that the Jevons paradox is actually occurring.
What to check going forward is how much AI usage grows as token prices fall, and whether that growth outpaces improvements in GPU processing efficiency. Alongside processing efficiency by use case and GPU utilization, it will also matter whether the results AI produces justify the additional spending.
If cheaper AI creates new tasks and services, and demand growth exceeds the headroom created by efficiency gains, NVIDIA can turn AI's falling prices into the next expansion of GPU sales.
Will cheaper AI mean even more GPUs are needed? The answer depends on how far AI usage grows and whether it translates into actual capital investment.