The focus of AI memory is shifting from simply making HBM faster to a new question: where should data that doesn't fit in HBM live on the GPU? On September 2, 2026, DigiTimes reported that Micron is exploring a concept in which high-endurance NAND would sit inside the GPU package or on the GPU board itself, allowing hundreds of gigabytes of data to be handled directly by the GPU.
The report followed a presentation by Micron's Mark Kiehlbauch at SEMICON Taiwan on September 1. However, Micron has not disclosed a product name or specification sheet. There has been no announcement of sample shipments or customer adoption. At this stage, what has surfaced is simply a design direction aimed at moving NAND closer to the GPU.
Placing Hundreds of Gigabytes of NAND Near the GPU
The core idea is to connect NAND to the GPU through a shorter path than today's NVMe SSDs. According to DigiTimes, Micron would sacrifice some storage density in NAND to boost I/O, bandwidth, and endurance instead. Under consideration are placing the NAND inside the same package as the GPU, or on a board positioned near the GPU.
This layer would handle data that doesn't require HBM-level responsiveness but becomes too costly to move if stored on distant storage. Candidates include LLM weights, reusable inference data, and KV caches. What exactly would be placed there hasn't been disclosed, and the NAND characteristics needed differ significantly between read-heavy weights and continuously written KV caches.
In an official explainer published in August 2026, Micron positioned HBM as the hottest data closest to the compute core, DRAM as the large working set, and NAND as the layer responsible for persistence and scale. By Micron's own estimates, a 256K-token context uses roughly 22GB of KV cache per session. That scales to about 1.4TB for 64 sessions, roughly 22TB for 1,000 sessions, and up to about 90TB if 1,000 sessions each use 1M tokens. These figures depend on the model and precision used, but they illustrate why HBM capacity alone cannot cover the full workload.
A Gap Between Micron's HBM4 and Its SSDs
Lining up Micron's actual AI products currently in mass production clarifies exactly what gap this GPU-adjacent NAND concept is targeting.
| Layer / Approach | Status as of September 3, 2026 | Key Published Specs | Path to GPU |
|---|---|---|---|
| Micron HBM4 | In mass production | 12-layer, 36GB, over 2.8TB/s | Wide connection inside GPU package |
| Micron's GPU-Adjacent NAND | Under exploration | Reported to be in the hundreds-of-GB range; bandwidth and latency undisclosed | Under consideration for in-package or on-board placement |
| Micron 9650 SSD | In mass production | Up to 28GB/s read, 5.5M IOPS | Via PCIe 6.0 bus and NVMe |
| Micron 9650 with NVIDIA SCADA | PoC possible with commercial parts | Demoed 230M IOPS on a single server | GPU initiates storage operations |
Comparing HBM4's over-2.8TB/s to the SSD's 28GB/s as simply "100x" would be misleading. The former is bandwidth per stack, while the latter is sequential drive read speed—access granularity and latency differ fundamentally between the two. Likewise, SCADA's 230M IOPS figure isn't for a single SSD but a demo result for an entire server.
Even so, the gap between the two ends is substantial. SCADA reduces CPU-mediated storage operations, but it doesn't physically relocate the NAND itself next to the GPU. GPU-adjacent NAND aims to change the physical placement itself, rather than simply improving the protocol.
HBF Leads on Both Specification and Prototyping
The existing concept closest to this goal is High Bandwidth Flash (HBF), led primarily by Sandisk and SK hynix. The first specification, published by OCP in August 2026, defines UCIe-based host connectivity, 8-layer or 16-layer stacks reaching up to 512GB, and bandwidth ranging from roughly 0.4 to 3.0TB/s depending on grade. It also covers electrical requirements for the connection, on-package reliability, and software I/O guidelines. Google and Tenstorrent are also listed as members.
Sandisk has gone further, disclosing in investor materials that it has taped out its first HBF memory die and plans to sample inference-oriented products in 2027. A tape-out marks the handoff of design data to a fabrication plant—it does not mean mass production of actual hardware has begun. Still, the timeline from specification to prototype has been made public.
Micron's GPU-adjacent NAND has only been reported as far as a placement concept in the hundreds-of-GB range. Beyond bandwidth and latency, the connectivity standard and sample timing remain undisclosed. It's also unclear whether Micron will join the OCP HBF effort or choose a different interface and memory model. While the two share the same target market—"between HBM and SSD"—compatibility between them cannot be confirmed.
Higher Bandwidth Doesn't Automatically Shrink Response Times
The real-world impact of GPU-adjacent NAND depends less on device bandwidth and more on whether software can intelligently select which data to move. A preprint from August 2026 examining HBF's practical utility extended TokenSim using four two-hour Qwen-Bailian production traces, five different models, and performance profiles for both H100 and B200 GPUs. This was not a measurement of an actual HBF product.
In simulations where HBF was simply substituted in place of SSD, average end-to-end latency increased by 2 to 5.5 times, and maximum SLO-compliant throughput dropped to between 1/2.7 and 1/1.1 of the SSD baseline. Even when the raw latency of the NAND medium itself was improved by 3.75x, overall workload response time improved by less than 1%.
The paper identified three underlying mechanisms. In the full HBF-1 and HBF-2 configurations, near-GPU memory capacity and bandwidth had to be traded off against HBF, reducing the number of requests the system could accept. Additionally, the existing two-tier KV hierarchy kept highly reused data in the near-GPU layer while sending less-reused, write-heavy data to HBF. Across all traces, writes exceeded reads, triggering thermal throttling before peak load was even reached.
What the paper rejects is not HBF itself, but rather inserting it directly into existing SSD-style KV hierarchies without modification. The authors identified reuse-aware placement, along with managing write volume and thermal load, as prerequisites for HBF to be viable. This is not a performance prediction for Micron's concept, nor has Micron stated it will adopt the same HBF configuration. Still, the trade-off against near-GPU memory, the read/write ratio, and thermal and lifespan characteristics are all verification items Micron will need to address with real hardware.
Four Disclosures Needed to Enter the Product Race
For Micron's statements to translate into genuine participation in the HBF market, four disclosures are needed. First is the connectivity method—whether UCIe or a proprietary approach—along with the implementation location, whether inside the GPU package or on the board. Second is real-world performance and endurance: sequential bandwidth and small-unit read latency need to be measured separately, along with bandwidth and power consumption under concurrent execution. Lifespan and thermal figures also need verification separately for read-heavy and write-heavy workloads.
Third is software. A published implementation is needed showing whether weights or KV cache would be placed on this layer, and when data movement between HBM and NAND would be triggered. Fourth is evidence of productization—sample timing, system partners, and customers need to be identified. Until all four of these points are in place, "capable of running large LLMs" remains a design goal to be solved, not a proven product outcome.
Scaling up HBM production and speed alone cannot easily address the demand for terabytes to tens of terabytes generated by long contexts and numerous concurrent sessions. The gap Micron is trying to enter genuinely exists. Whether that gap gets filled by NAND packaging, data placement software, or a combination of both remains to be seen. In future announcements, the deciding factor won't be capacity alone—it will be exactly what data gets moved, and under what conditions.