d-Matrix's Raptor, an AI inference accelerator built on 3D-DRAM, has moved from concept to measured silicon. Around the time of its Hot Chips tutorial on August 23, 2026, Wccftech ran a headline claiming "1/10th of HBM." But what the peer-reviewed ISCA 2026 paper actually confirms is roughly 105 TB/s of card-level bandwidth, an average flit latency of about 2.5 ns, and an I/O energy of 0.376 pJ/bit—the last figure measured under limited conditions—all taken from the Raptor test chip.
Not all of these numbers come from the same kind of measurement. Bandwidth, latency, I/O energy, and refresh behavior were measured on real silicon. Meanwhile, the reported multiples for LLM throughput per card (tok/s/card) and time-per-output-token (TPOT) come from architectural modeling that pairs the same 10 PFLOPS XPU with different memory configurations. The paper does not establish how many times better Raptor performs than an HBM-based GPU as a shipping product.
Raptor's goal is to speed up the repeated reads of weights and KV cache during decoding while cutting transfer power. But commercialization still requires managing heat and redundancy, and securing sufficient capacity and yield. Software support is also needed. This paper shows that these constraints were factored into the design—nothing more.
The Gap Between 105 TB/s and "1/10th of HBM"
According to the ISCA paper, Raptor measured approximately 105 TB/s of card-level bandwidth and an average flit latency of about 2.5 ns at 700 MHz. The paper describes this as 12.5x an HBM3 card's bandwidth, or 6.25x if HBM4 is assumed to double HBM3's bandwidth. That 100 TB/s figure isn't per-stack or per-chiplet—it's the bandwidth aggregated across the entire card.
I/O energy also changes meaning depending on measurement conditions. A figure of 0.455 pJ/bit at 100% switching was reduced by 18% to 0.376 pJ/bit using stream flipping, which caps the switching rate at 40–48%. This measurement was taken at VDD1=2.5V, VDD11=1.1V, 500 MHz, read-only operation, BL128, and 8 active banks—conditions that must be read separately from the 700 MHz design target.
Wccftech's "1/10th of HBM" claim would hold if comparing the 0.3 pJ/bit target disclosed in 2025 against an HBM4 figure of roughly 3 pJ/bit. But for the 2026 measured value of 0.376 pJ/bit, the paper itself compares this against a representative HBM3 figure, yielding roughly a 6x reduction. Micron's published HBM4 page doesn't disclose an absolute pJ/bit figure, so it can't be used for a direct comparison with current HBM4 products.
A simple calculation shows that transferring 100 TB/s at 0.376 pJ/bit would require roughly 301 W for I/O alone. This is neither the card's total power consumption nor a tokens/joule figure—rather, it indicates how well transfer-related power can be controlled as bandwidth increases.
Packing 840 Banks Into 256 Channels
A Raptor card consists of up to 4 MCMs, each containing 4 chiplets. Each chiplet is designed with a TSMC N4P logic die face-bonded to a 3D-DRAM die at a 36 µm pitch, operating at 1.2 GHz. Each chiplet houses 840 DRAM banks, 256 channels, and 256 tensor engines. 72 banks are reserved as spares, leaving 768 banks used at 3 banks per channel.
Simply adding more banks doesn't translate into usable bandwidth. Raptor's stream blocking reads 96 B at a time from 3 banks, then assembles 128 B flits using a 96 B partial buffer and a fixed 2-access pattern—a configuration designed to avoid wasted reads. Stream flipping, meanwhile, reduces bit-flipping to lower I/O energy.
Aligning the units of comparison reveals the design trade-offs. The paper's baseline 3D-DRAM configuration delivers 32 GB and 100 TB/s per card. The comparison HBM configuration offers 192 GB and 18 TB/s, while the SRAM configuration offers 4 GB and 150 TB/s. Micron's HBM4 12-High stack delivers 36 GB and over 2.8 TB/s per stack, while NVIDIA's provisional Rubin GPU specification lists 288 GB of HBM4 and 22 TB/s. Raptor trades capacity for a much larger aggregated bandwidth per card compared to HBM.
Building Heat and Defect Management Into the Design
Stacking logic on top of DRAM turns heat into a problem that can't be solved by cooling alone after the fact. Raptor uses a 4 ms refresh interval at a 105°C junction temperature—8 times more frequent than HBM's standard 32 ms. By finely partitioning the 840 banks into 1,364 rows per bank, the paper estimates a bandwidth loss from refresh of just 1.37% at 700 MHz.
However, the thermal figures mix measurement with analysis. Based on an assumed 422 W per MCM (about 106 W per chiplet), the thermal analysis found that an optimized heatsink kept peak temperatures around 93°C; standard air cooling could stay under 105°C only up to 63 W per chiplet; and liquid cooling kept temperatures below 60°C at 106 W. This is an analytical model, not a measured thermal design value for a shipping card.
For reliability, bank chaining distributes the 72 spare banks to route around defective banks while preserving channel width. Error correction relies on interleaved [144,140] Reed-Solomon codewords with background scrubbing. While the outlines of a production-oriented design are emerging, no absolute yield figures have been published. How far these design measures translate into actual production yield remains unverified.
The Conditions Behind the 4.71x Figure
Across Llama-3.1 70B, DeepSeek-V3, Kimi K2, GPT-OSS, Whisper, and Canaria, the paper reports an average 4.71x improvement in tok/s/card relative to an HBM baseline, with TPOT at 1/9.96th of the HBM baseline. Throughput relative to an SRAM baseline was 2.44x. Taken at face value, these numbers suggest Raptor's bandwidth design substantially changes inference latency.
But the evaluation conditions are explicit. This is an architectural model pairing the same 10 PFLOPS XPU with different memory configurations, using a network latency of 0.5 µs and a bandwidth of 1 TB/s. Batch simulations using a Poisson arrival process ran for one second, with a representative load of 110 requests/second and a batch size of 32. This is not a hardware comparison against commercially available HBM GPUs or SRAM accelerators.
The paper's results also show the advantage isn't uniform. At a 4K context length with 0.5 µs network latency and 1 TB/s bandwidth, 3D-DRAM showed 4.38x the tok/s/card of HBM and 3.15x that of SRAM—but the advantage shrinks at higher latency due to collective communication overhead. For a speech model using a 448-token context, all memory configurations fit on a single card, and SRAM was fastest, followed by 3D-DRAM, then HBM.
Raptor's design isn't claimed to deliver peak performance across every inference scenario. Rather, the evaluation suggests that the bandwidth and latency advantages matter most when weight and KV cache transfers dominate—conditions more likely with long contexts or large models that span multiple cards.
The Commercial Verdict Comes Before Any HBM Replacement
d-Matrix states it intends to commercially deploy 3DIMC through Raptor. The paper notes that 2-High and 4-High chips are currently under lab testing, but the 200/400 TB/s and 64/128 GB figures cited are design points used for ablation studies—not commercial specifications. No release date, pricing, or production timeline has been disclosed. Customers, third-party benchmarks, and absolute yield figures also remain unknown.
At 32 GB, Raptor's 3D-DRAM capacity is smaller than the 288 GB of HBM4 in Rubin's provisional specification. How large models would be accommodated is left to solutions such as multiple cards, a 128 GB LPDDR5X secondary memory tier per MCM, or combinations of heterogeneous accelerators. The paper proposes one arrangement where a GPU handles attention while Raptor handles the FFN layers that mainly read expert weights. Another proposal has Raptor handling memory-bound draft generation in speculative decoding, with a GPU handling verification. Both remain deployment proposals, not commercial track records.
What Raptor needs to clear next isn't the 105 TB/s measurement itself. Only once production cards are running at known power and cooling levels, carrying specific models at specific prices, and holding up under independent hardware comparisons will 3D-DRAM be positioned as a genuinely distinct inference memory option alongside HBM.