Chinese technology companies bought more than 1 million of NVIDIA's gaming and workstation GPUs in 2026, the Financial Times (FT) reported on October 13. Production of domestic AI chips is rising in China, but it is not enough to meet surging demand for computing power, so companies are reportedly repurposing GPUs originally designed for other uses to handle AI inference. Another factor is a desire to keep using the existing software environment built around NVIDIA's CUDA. However, the number of GPUs purchased does not translate directly into an equivalent increase in AI processing capacity. AI inference has a stage that processes input and a stage that generates answers, and each demands different hardware characteristics. Understanding this difference is key to seeing what demand the mass purchases can meet and where their limits lie.
Why Chinese companies keep buying NVIDIA GPUs even as domestic AI chip output grows
According to the FT report, the products Chinese companies bought include the RTX PRO 6000D, RTX PRO 5000 and RTX 5090. All are GPUs mainly intended for gaming, design and rendering, but Chinese companies are reportedly using them for part of the inference workload of AI models. Note that the purchase volumes and specific uses come from the FT's reporting and are not official sales statistics published by NVIDIA.
Behind this trend is the fact that, although domestic AI chip production in China is expanding rapidly, demand is growing even faster. According to estimates relayed by the FT, the supply of AI chips available to Chinese companies in 2026 is expected to reach 3.5 million units, of which roughly 1 million will be manufactured overseas. Demand, meanwhile, is projected to far exceed 5 million units. It would therefore be inaccurate to read the 3.5 million figure as the number of chips made entirely in China.
China's AI chip output has also reportedly more than doubled from the previous year. So this is not a simple story in which continued reliance on NVIDIA GPUs stems from stalled domestic chip development or production. Domestic supply capacity is growing while demand for AI computing keeps rising, and companies appear to be forced to use whatever hardware is available, matched to the task.
Still, simply adding the number of purchased gaming GPUs to the number of domestic AI chips does not show how far the supply shortfall has been eased. GPUs differ in compute performance, memory capacity and supported software, and not every purchased unit is necessarily running in an actual AI service. What the report shows is the scale of procurement; it does not establish that buying 1 million units secured a comparable amount of new AI processing capacity.
Prefill and decode place different demands on GPUs in AI inference
One of the uses for the repurposed GPUs reported by the FT is the "prefill" stage of AI inference. Large language model (LLM) inference proceeds in two broad stages: prefill and decode. In prefill, the model processes the user's input all at once and prepares the computation results needed to generate a response. In decode, the model then generates the answer bit by bit, drawing on those results. Both stages are essential, but the performance each requires from a GPU differs because of their processing characteristics.
In a design document for its inference platform Dynamo, NVIDIA explains that prefill is generally constrained mainly by compute performance, while decode is generally constrained mainly by memory performance. In prefill, many input tokens can be computed together, making it easier to use the GPU's arithmetic units efficiently. In decode, each newly generated token in sequence requires referencing the model's parameters and information about past context, so memory capacity and bandwidth tend to determine performance.
Given this difference, it would be wrong to interpret the idea of repurposing gaming GPUs for relatively simple AI work as assigning them light computational tasks. Especially when processing long documents or large volumes of input data at once, prefill also requires considerable computing power. Even if such GPUs cannot replace everything handled by cutting-edge AI accelerators, repurposing them makes sense if they can process specific stages such as prefill efficiently.
NVIDIA's Dynamo also supports "disaggregated serving," which assigns prefill and decode to different GPUs. In this setup, the results of past input computation generated during prefill, known as the "KV cache," are transferred to the GPUs handling decode. This lets operators use hardware suited to each stage, but if transferring the KV cache takes too long, the benefit of separating the GPUs can be lost. It is therefore necessary to evaluate efficiency across the whole system, including memory capacity and communication speed, rather than the GPU's compute performance alone. That said, this describes how disaggregated inference works in general; it has not been confirmed that the Chinese companies in the FT report use such a configuration.
Reliance on CUDA supports demand for NVIDIA GPUs, even as domestic stacks advance
Hardware shortages are not the only reason Chinese companies keep buying NVIDIA GPUs. The FT also cites CUDA, NVIDIA's software platform for its GPUs, as an important factor. CUDA is a development and runtime environment for parallel computing on NVIDIA GPUs, and a wide range of software used for AI training and inference is built on the assumption of this environment.
For a company already using CUDA, moving to domestic AI chips requires not only securing hardware but also adapting existing programs and processing infrastructure to the new environment. If it can keep using NVIDIA GPUs, it may be able to meet rising computing demand while making use of its existing software assets. However, the migration costs and time required for each company are not known, so it cannot be said that using NVIDIA GPUs is always cheaper.
Software environments for Chinese AI chips are also being developed to address these challenges. At HUAWEI CONNECT 2026 on September 17, 2026, Huawei announced that CANN, the software platform for its Ascend processors, has shifted to ongoing community-driven open-source development. It also cited support for more than 90 external open-source projects, including PyTorch and vLLM.
But more supported software is a separate matter from being able to run existing CUDA programs with the same effort or performance. Huawei's announcement is the company's own account of an expanding Ascend software environment, not a verified comparison of migration costs or execution performance against CUDA. Likewise, the report that NVIDIA GPU purchases continue does not directly show whether China's push for domestic AI chips is succeeding or failing; it appears to reflect a situation in which hardware supply is growing and software migration is progressing in parallel.
The value of 1 million GPUs depends on the workloads they handle
The reported purchase scale of more than 1 million units alone does not tell us how far Chinese companies can now meet their AI computing demand. Prefill, decode and the data transfer between them are each limited by different factors. Based on the mechanisms described in the "Disaggregated Serving," "Efficient KV Transfer" and "Performance Characteristics" sections of NVIDIA's technical documentation on Dynamo, the points to check at each stage can be summarized as follows.
| Stage | Key resources and conditions | What to check in performance evaluation |
|---|---|---|
| Prefill (input processing) | Compute performance, input length | Can it shorten the time to process the expected input and produce the first response? |
| Decode (answer generation) | Memory capacity and bandwidth, KV cache capacity | Can it generate multiple responses at the required speed while holding the model and past context? |
| Data transfer between stages | KV cache transfer speed, communication load | Is the benefit of separating the stages lost to waiting time from data transfer? |
This breakdown summarizes general characteristics of inference for the same language model and does not rank the performance of specific GPUs. Which resource actually becomes the bottleneck varies with model size, quantization method, input length, the number of concurrent requests and other factors.
For example, even a professional GPU with a large amount of memory is not necessarily suited to every inference workload. NVIDIA's published provisional specifications for the RTX PRO 5500 list 84GB of ECC-capable GDDR7 memory, 1,398GB/s of memory bandwidth and maximum power consumption of up to 600W. Memory capacity determines whether the model and context information can be held, while memory bandwidth affects how fast data can be read and written. At the same time, given the 600W maximum power draw, power supply and cooling capacity must also be considered when running large numbers of GPUs.
NVIDIA lists LLM inference as a use for the RTX PRO 5500, but the product's specifications are subject to change, and its availability is described only as "coming soon." The publication of the specifications does not mean that sales to China have been approved or that a specific shipping date has been set.
The mass purchase of NVIDIA GPUs by Chinese companies highlights that increased domestic AI chip production alone has not met surging computing demand. But how much supply this procurement actually translates into depends not on the number of GPUs but on which workloads they are used for and how fast and efficiently they run. Going forward, measured performance in both prefill and decode, along with system-wide efficiency including data transfer and power consumption, will be important for judging how practical such GPU repurposing is.