On November 9, 2026, Google announced two new additions to its eighth-generation TPU line of in-house AI accelerators: the "TPU 8tx" and the "TPU 8i Max." Both build on the "TPU 8t" and "TPU 8i" announced in April. The 8tx can protect sensitive data while it is being processed, and the 8i Max puts two 8i chips into a single package to expand compute and memory-related resources. As generative AI keeps long conversation histories and uses external tools to carry out complex tasks, overall system performance depends not only on compute power but also on memory capacity and bandwidth, chip-to-chip communication, and data security. Rather than developing a new compute chip, Google is trying to address these challenges by optimizing existing designs for specific uses. However, figures such as the "2x" announced this time do not necessarily mean real AI applications will run faster by the same proportion.
The eighth-generation TPU grows to four products, with differences in system configuration
The TPU 8tx and TPU 8i Max both make use of existing eighth-generation chips. The 8tx uses the same silicon as the 8t, but the host CPU changes from Google's own Axion to a fifth-generation AMD EPYC, and TPU-to-TPU connectivity uses Boardfly. The 8i Max, meanwhile, places two existing 8i chips in a single package. In other words, Google has not overhauled the compute chip architecture itself; it has changed the chip combinations and surrounding systems to serve different uses.
The existing 8t and 8i were also designed with different uses in mind from the start. The 8t, aimed at training large AI models, uses Google's Axion CPU as host and a 3D torus interconnect between TPUs. The 8i, designed for inference, combines the Axion CPU with Boardfly. Boardfly and the 3D torus both handle data communication between TPUs, but the connection method and overall system configuration, including the host CPU, differ. With these additions, Google now offers four options by use case: large-scale training, general inference, confidential inference, and long-context inference.
Based on Google's official November 9 announcement and the technical deep dive on the 8t and 8i published in April, the configuration and intended use of each system break down as follows.
| System | Underlying chip | Main configuration and features | Intended use | Caveats for performance comparisons |
|---|---|---|---|---|
| TPU 8t | 8t | Axion CPU, 3D torus interconnect | Large-scale AI model training | Uses the same silicon as the 8tx, but the system configuration differs |
| TPU 8i | 8i | Axion CPU, Boardfly interconnect | AI inference | The baseline for the 8i Max resource multiples |
| TPU 8tx | Same silicon as 8t | Fifth-generation AMD EPYC, Boardfly interconnect, protects data and model weights during processing | Confidential inference, small-scale training | Google claims up to 50% lower latency than the 8t. Detailed measurement conditions have not been disclosed |
| TPU 8i Max | Two 8i chips | Single package; compute, SRAM capacity, and HBM bandwidth each expanded to 2x that of the 8i | Long-context AI inference | The 2x is a hardware resource spec multiple, not actual response speed |
Note that the performance multiples published for each product mean different things. The 8i Max's "2x" refers to the per-package multiple of compute and memory-related resources compared with a standard 8i. The 8tx's "up to 50% lower latency," by contrast, is Google's performance claim against the 8t, not a figure showing the amount of hardware resources. Neither comes from a benchmark measuring all four products under the same conditions, and the announcement text does not disclose the absolute hardware specifications of the 8i Max.
How much faster does the TPU 8i Max's "2x" make AI inference?
The 8i Max's defining feature is that putting two 8i chips in one package doubles compute, SRAM capacity, and HBM bandwidth relative to a standard 8i. SRAM is high-speed memory placed close to the compute units, while HBM is used to quickly feed AI model weights and intermediate calculation data. In the technical documentation for the 8i that Google published in April, SRAM capacity was given as 384MB and HBM bandwidth as 8,601GB/s. Google says the 8i Max increases the number of chips per package while keeping the same rack footprint as before.
However, a doubling of HBM bandwidth and a doubling of HBM capacity are different things. Memory bandwidth is the amount of data that can be transferred per unit of time, while memory capacity is the total amount of data that can be held. What Google says has doubled this time is HBM bandwidth; it has not announced that HBM capacity, or the context length an AI model can handle, has doubled. Also, if both compute and HBM bandwidth double, memory bandwidth per unit of compute, calculated simply, stays the same. This is a relationship derived from the published spec multiples, not a measured result showing that the so-called "memory wall," where data supply from memory to compute units limits performance, has been removed.
In generative AI handling long contexts, the "KV cache," which stores the computation results for previously input and generated tokens, becomes important in addition to model weights. Reusing the KV cache removes the need to repeat calculations for past tokens and makes subsequent token generation more efficient. At the same time, the longer the conversation history or input text, the larger the KV cache that must be held, and the greater the demands on memory capacity and bandwidth. Adding compute units alone is therefore not enough; performance depends on how efficiently the necessary data can be supplied and where the already computed state is kept.
With the 8i Max, Google aims to expand, at the package level, the compute resources and fast memory access needed for long-context inference. But doubling hardware resources does not mean the response speed of an AI application will uniformly double. The actual speedup varies with model size, input length, the number of concurrent requests, memory utilization efficiency, and other factors. The announcement did not provide processing speeds for applications running on the 8i Max or the power consumption of an entire rack. Even if the same rack footprint can be used, whether deployment efficiency improves when power supply and cooling are included will need to be verified separately.
TPU 8tx is aimed at enterprise AI that handles sensitive information
The TPU 8tx was developed mainly for AI inference involving highly sensitive data. Its key feature is support for "confidential computing," in which hardware protects and isolates data and AI model weights during inference. In general, handling confidential information is a challenge not only for protection at rest and in transit but also for security while the data is actually being processed. By supporting protection during processing, the 8tx aims to widen the options for companies that want to let AI use sensitive information.
To achieve this, Google combined the same compute chip as the existing 8t with a fifth-generation AMD EPYC CPU and Boardfly interconnect. Its main use is confidential inference, but it also supports training of small-scale AI models. Specifically, it is intended for processes that adapt a model to a particular purpose, such as additional training, distillation, and reinforcement learning. By using silicon built for large-scale training while changing the host CPU, communication method, and confidentiality protections, Google has made a configuration that can handle both inference and small-scale training on the same system.
On performance, Google says the 8tx cuts latency by up to 50% compared with the 8t. However, conditions such as the AI model used for comparison, input length, and degree of parallelism are not stated in the announcement text. Google also says enabling confidential computing causes almost no performance penalty, but that is a claim about the overhead of the confidentiality features. It is a separate assessment from the latency reduction versus the 8t, and does not guarantee that both benefits will be obtained at the same time in a given workload.
The specific scope of the confidentiality protection also needs attention. The announcement alone does not let us confirm detailed implementation conditions, such as the boundaries of hardware isolation, how encryption keys are managed, or the authentication mechanism. And even if data being processed is protected by hardware, operational questions, such as whose requests an application permits and what information is recorded in operation logs, are not automatically resolved. Whether a company can let AI handle confidential information has to be judged by combining hardware protections with application-side access control, log management, and similar measures.
Software also tackles the "memory wall," speeding up KV cache transfers
Alongside expanding its hardware, Google is developing software to ease memory constraints in AI inference. One example is "TPU Sync," a C++ library that moves KV cache and AI model weights between different levels of the memory hierarchy. According to Google's technical explanation, TPU Sync combines not only HBM but also host-side DRAM and shared storage, and moves the necessary data to the appropriate place. It uses asynchronous DMA for transfers and also supports data movement over the network. However, this is a technology for using memory efficiently; it does not increase the physical capacity of HBM or the maximum context length an AI model itself supports.
Generative AI inference consists broadly of two stages: "Prefill" and "Decode." Prefill processes the input prompt all at once, while Decode generates the response tokens one after another using those results. Because the two stages place different demands on compute resources, some setups assign each to a separate accelerator. In this approach, however, the KV cache generated in Prefill must be transferred to the Decode side, and if the transfer takes long, the benefit of separating the compute resources is lost. It particularly affects TTFT (Time to First Token), the time until the first token is returned to the user.
In internal tests published by Google, TPU Sync reduced the median KV cache transfer time for Qwen3 235B from several seconds to about 230 milliseconds. Google also said that when the same model was run under low-concurrency conditions, average TTFT fell by 60%. The result shows that speeding up KV cache handoff can shorten the wait until a response begins. However, the tests used the previous-generation TPU v7x and did not measure the performance of the newly announced TPU 8i Max.
Google also ran a test that reuses computation results corresponding to a shared prompt prefix. With a 64K-token context, it reported average TTFT reduced by up to 90% and throughput improved by more than 50%. This is the result when the state of a previously computed common portion can be reused, and the same improvement will not be obtained for every conversation or request. Because the conditions also differ from the earlier KV cache transfer test, the improvement rates cannot be compared directly or combined to estimate a performance gain.
These tests were run in July 2026 on TPU v7x in a 2×2×1 configuration. Connectivity was PCIe Gen5 x16 per chip and one or two 200Gbps NICs per host, and the software used was tpu-inference v0.24.0, JAX 0.10.2, and libtpu 0.0.42.1. When evaluating the reported speedups, therefore, measurement conditions such as hardware generation, network configuration, and degree of parallelism need to be taken into account. Note that some of TPU Sync's storage-related features and its reinforcement learning module are currently experimental.
Google is also addressing processing specific to AI agents from the software side. Because agents spend time not only on inference but also on waiting for requests to external tools and their responses, keeping compute resources running continuously is not always efficient. "Agent Compute Engine," introduced in the November 9 announcement, can pause sessions that are waiting and resume them when needed. As for the availability of software announced at the same time, TPU agent suite is in preview, while Cloud Pathways and GKE dynamic slicing are generally available. These, however, describe the status of the software, and do not mean the TPU 8tx or 8i Max will be generally available in all regions.
What Google is working on is not simply building high-performance TPUs. Along with adding chips to expand compute and memory bandwidth, it is building mechanisms to transfer and reuse already computed state efficiently and to make good use of compute resources while waiting on external tools. What matters is that it is trying to improve the efficiency of the whole AI inference system from both the hardware and software sides.
Competition with NVIDIA turns on software portability as well as performance
According to Google, nearly 100 companies use TPUs, and it is also working on deployments in customers' own data centers. As the number of users grows, the burden of moving from an existing AI development environment to TPUs becomes more important. No matter how high-performance the hardware, it becomes a barrier to adoption if existing AI models and development tools must be extensively rewritten. Google's answer is an effort to improve compatibility with PyTorch.
On October 21, Google released as open source "TorchTPU," developed jointly with the PyTorch team. Its aim is to let existing PyTorch workloads move to TPUs with minimal code changes. Developers can keep their familiar workflow of checking program behavior in Eager Mode and then optimizing with torch.compile. However, standard PyTorch APIs working is a separate matter from an application containing custom GPU-specific implementations running at sufficient performance as is. Beyond porting code, validating performance on real workloads and optimizing where necessary remains to be done.
Google's published figures on performance also need to be understood correctly. When it announced the eighth-generation TPUs in April, the company said price-performance versus the previous-generation Ironwood improved by up to 2.7x for the 8t and by up to 80% for the 8i. But these are comparisons with Google's own previous-generation products, not results of measuring NVIDIA GPUs under the same conditions. Nor has any common-condition benchmark been published showing performance advantages of the newly announced 8tx or 8i Max over NVIDIA products. These numbers alone therefore cannot support a conclusion that TPUs are higher-performing or more efficient than NVIDIA GPUs.
What stands out in this announcement is that Google is trying to remove a wide range of constraints on deploying AI systems, not just raise compute performance. The 8tx addresses protection of confidential data, the 8i Max expands the hardware resources needed for long-context inference, and TPU Sync and TorchTPU aim to reduce the burden of memory management and software migration. This looks like a strategy to broaden TPU adoption not by competing on peak compute performance alone, but by building an environment in which companies can easily migrate and operate their existing AI systems.
Still, judging that advantage requires comparisons under conditions close to real workloads. It is necessary to verify how much processing capacity is obtained when the AI model, context length, degree of parallelism, output quality, and target response time are all matched. Total cost also matters, including not just hardware usage fees but migration from existing systems, operations management, and power supply and cooling. The announcement did not confirm pricing, availability regions, or a clear general availability date for the new products, so specific deployment conditions will have to await further disclosure.
Google's expansion of the eighth-generation TPU to four products shows that competition among AI accelerators is broadening from the compute performance of individual chips to the efficiency and ease of deployment of whole systems. What ultimately shapes enterprise adoption decisions is how cheaply, including migration and operations, an AI service can be delivered at the same output quality and response time. How much of an advantage the two new products have on that score can be assessed once detailed specifications, measured data, and availability terms are made public.