Groq is considering 3D stacking of DRAM for its next-generation AI chip, ZDNet Korea reported on October 20, 2026. That could move the company away from a design built around fast but capacity-limited SRAM and toward larger memory. The report relies on anonymous industry sources, however, and is not an announcement of a product that has been adopted. The current NVIDIA Groq 3 LPX already has 12TB of DDR5 memory per rack. So the significance of adding stacked DRAM lies not only in increasing capacity, but also in how it might change the path that delivers data to the compute circuits.
Reports of 3DS under consideration, and the stacking target still unclear
A semiconductor industry source who spoke to ZDNet Korea said Groq will introduce 3D packaging in its next-generation chip, and that it "involves stacking DRAM rather than the conventional SRAM." Another source analyzed the move as consistent with market demand for large capacity. Both accounts are anonymous; neither Groq nor NVIDIA has published specifications.
The article cited a method called "3DS (3D Stacked Memory)." DRAM dies, the small pieces of semiconductor, are stacked vertically and connected by through-silicon vias (TSVs), electrodes that run through the silicon. ZDNet explains that while HBM also uses TSVs, this 3DS approach emphasizes increasing capacity while retaining the conventional DRAM interface. Stacking alone does not mean it would deliver the same bandwidth as HBM.
The report does not make clear whether the DRAM would sit directly on top of the compute chip, or how much SRAM would be kept. The memory capacity and release timing of the next generation have not been disclosed either, so it is too early to conclude that SRAM will be abolished entirely.
The relationship between Groq and NVIDIA also needs to be kept distinct. Groq's announcement of December 24, 2025 described a non-exclusive licensing agreement for inference technology and the move of founder Jonathan Ross and others to NVIDIA. Groq said at the time that it would continue to operate as an independent company and that GroqCloud would continue. Framing this as that agreement plus a transfer of technology and people, rather than as an acquisition of the whole company, avoids confusing NVIDIA's Groq 3 products with Groq's cloud business.
Speed backed by 500MB of SRAM, and expansion across chips
According to NVIDIA's technical explanation, each current Groq 3 LPU has 500MB of on-chip SRAM with bandwidth of 150TB/s. An LPX rack holds 256 of them, for a total SRAM capacity of 128GB and aggregate SRAM bandwidth of 40PB/s. The design reads large amounts of data from right next to the compute circuits.
This SRAM differs in character from a typical CPU cache, where hardware decides what data moves in and out. The compiler and runtime software place data such as model weights and intermediate results, and they explicitly determine when to transfer it. It is a mechanism that plans in advance when each computation runs and where the data it needs is brought from. NVIDIA says this kind of control reduces unpredictable processing stalls and keeps response times stable.
Still, 500MB is not designed to hold an entire large model. A model that does not fit in SRAM is spread across multiple LPUs, for example by splitting it by layer. Communication between chips therefore also underpins performance. The chip-to-chip interconnect bandwidth for the whole rack is given as 640TB/s, but this is a separate metric from the 40PB/s of SRAM bandwidth. Lumping together how fast data is read from memory and how fast it is moved across chips would also lead to misjudging the effect of adding capacity.
There is also a division of labor with GPUs. In configurations that combine Vera Rubin and LPX, the Rubin GPU handles reading the input and the attention mechanism during generation, while the LPU takes on the FFN (feed-forward network) computation layers and the expert portions of MoE models. Because intermediate data is exchanged at each generation step, the scale of model a system can handle cannot be determined from the SRAM capacity of a single LPU alone.
Current LPX already uses DRAM. Why stacking still matters
The NVIDIA LPX product page states that one rack has 128GB of SRAM and 12TB of DDR5. The technical explanation also describes configurations in which each tray can use up to 512GB of DRAM via a connectivity-expansion circuit and up to 256GB via the host CPU. The current LPX is not built entirely from SRAM across the rack.
The current on-chip SRAM, the external DDR5, and the 3DS in the reported plans differ in placement, role, and certainty. Lining up NVIDIA's official specifications and ZDNet's reporting not as substitutes of equal capacity but in terms of what is placed where makes the differences easier to see.
| Memory | Placement and role | What can be confirmed |
|---|---|---|
| Current on-chip SRAM | Placed inside the LPU; reads data used for inference at high speed | NVIDIA figures: 500MB per chip, 128GB per rack |
| Current DDR5 | Memory outside the LPU, used via a connectivity-expansion circuit or the host CPU | NVIDIA figure: 12TB per rack |
| 3DS reported as under consideration for next generation | Idea of connecting DRAM dies vertically to increase capacity; positional relationship to the compute chip unknown | ZDNet anonymous sourcing; specific capacity and bandwidth undisclosed |
This comparison shows that the current 12TB cannot simply be swapped for stacked DRAM and evaluated as equivalent. SRAM inside the LPU and external memory accessed through a separate circuit deliver data along different paths. Having large capacity is not the same as being able to access that capacity as fast as SRAM.
If stacked DRAM could place more data closer to the compute circuits, there may be room to ease the burden of splitting a model across many chips and increasing external communication. But that is a benefit inferred from the design, not an effect measured on a next-generation Groq chip. With the stacking location unknown at this stage, it is also impossible to estimate how much communication volume or how many LPUs would be saved.
Even with more capacity, compute has to wait if data does not arrive at the moment it is needed. To preserve the low latency Groq is known for, the question is how access to the new memory would fit into the schedule of computation and transfers that the compiler builds. An explanation that simply trades speed for capacity misses this design challenge.
Cerebras's combined approach shows a choice to keep SRAM
In the technical roadmap it presented at Hot Chips 2026, Cerebras outlined a concept for a future CS-6 that combines wafer-scale SRAM and compute circuits with 3D-stacked DRAM over ultra-high-bandwidth connections. The company's aim is to keep the advantage of placing data near the compute circuits while increasing the amount of model each system can hold.
This is not a concept for removing SRAM. It is a direction that uses SRAM's speed and supplements capacity with DRAM. However, the CS-6 is a future plan, and it has not been shown to use the same stacking structure as Groq. Cerebras's explanation cannot be used to corroborate Groq's specifications or performance; the only thing the two have in common is the attempt to use stacked DRAM to expand capacity.
When evaluating Groq's next announcement, it will be worth checking not only the capacity figures but also where the DRAM is placed and what division of labor it has with SRAM. If bandwidth and response latency are then shown when running actual models, it will be possible to judge how the capacity expansion translated into inference speed. If stacked DRAM lets more model data be kept close at hand without making computation wait, it could become a way to extend low-latency inference to larger models.