How High-Bandwidth Memory Works: Stacked DRAM, Wide Interfaces and Why It Is Hard to Make
High-bandwidth memory gets its speed from width, not clock rate: stacks of DRAM dies, joined by vertical connections, sit beside the processor and are read over a 1,024- to 2,048-bit interface. This post sets out how that works, what each generation changed and why more layers make it harder to build.
Checked against JEDEC's HBM3, HBM4 and SPHBM4 releases, the memory makers' own HBM pages and announcements, and Micron's 10-Q for the quarter ended May 28, 2026
- Memory dies and the base die in one stack
- Vertical and horizontal connections
As of Maker rows are vendor-stated; JEDEC rows are specification limits
| Entry | Source | Pin speed | Bandwidth per stack | Channels | Stack height and capacity | Figure |
|---|---|---|---|---|---|---|
| HBM3 (JEDEC JESD238, Jan 27, 2022) | JEDEC | Up to 6.4 Gb/s1 | 819 GB/s per device1 | 16 (32 pseudo-channels)1 | 4 GB (8 Gb, 4-high) to 64 GB (32 Gb, 16-high)1 | Standard limit |
| HBM4 (JEDEC JESD270-4, Apr 16, 2025) | JEDEC | Up to 8 Gb/s, 2,048-bit2 | Up to 2 TB/s2 | 32, each with 2 pseudo-channels2 | 4-, 8-, 12- or 16-high; up to 64 GB (32 Gb, 16-high)2 | Standard limit |
| SPHBM4 (JEDEC JESD330-4, Jul 13, 2026) | JEDEC | Not stated; 512 data signals with 4:1 serialization3 | Same aggregate throughput as HBM43 | Not stated | Same capacity per stack as HBM43 | Standard limit |
| HBM3E, SK hynix (volume production Mar 19, 2024) | SK hynix | Not stated | Up to 1.18 TB/s4 | Not stated | Not stated | Vendor-stated |
| HBM3E, Micron | Micron | More than 9.2 Gb/s5 | More than 1.2 TB/s per placement5 | 16 (32 pseudo-channels)5 | 24 GB (8-high), 36 GB (12-high)5 | Vendor-stated |
| HBM3E 12H, Samsung (announced Feb 27, 2024) | Samsung | Not stated | Up to 1,280 GB/s6 | Not stated | 36 GB, 12-layer6 | Vendor-stated |
| HBM4, SK hynix (Sep 12, 2025) | SK hynix | More than 10 Gb/s, 2,048 I/O7 | Not stated (“doubled” versus previous generation)7 | Not stated | Not stated | Vendor-stated |
| HBM4, Micron | Micron | More than 11 Gb/s, 2,048-pin8 | More than 2.8 TB/s per stack8 | Not stated | Not stated | Vendor-stated |
| HBM4, Samsung | Samsung | Up to 13 Gb/s9 | Not stated | Not stated | Not stated | Vendor-stated |
JEDEC's figures are limits in a standard, not measurements, and the makers' figures are their own claims about products; the two are kept in separate rows and are not ranked against each other. “Not stated” means the source page read does not give the figure. HBM3E is a maker-defined extension of HBM3 that JEDEC's releases read here do not cover. For which accelerators carry the most HBM, see AI Accelerators With the Most HBM per Chip. Figure labels: how they work.
How it works
What HBM is
Micron's filing describes high-bandwidth memory as a 3D stacked DRAM architecture that uses through-silicon via (TSV) connections for more efficient communication, which gives it higher bandwidth than other memory types.10 SK hynix describes the same idea as vertically interconnecting multiple DRAM chips, and counts six generations: the original HBM, HBM2, HBM2E, HBM3, HBM3E and HBM4.4
Bandwidth comes from width
A stack's bandwidth is the number of signals on its interface times the speed of each one. HBM4's JEDEC specification sets up to 8 Gb/s across a 2,048-bit interface for up to 2 TB/s per stack,2 and SK hynix says its HBM4 doubled bandwidth by adopting 2,048 I/O terminals, double the previous generation's.7 Our own arithmetic shows how the pieces fit: 2,048 signals at 8 Gb/s is 2,048 × 8 ÷ 8 = 2,048 GB/s, and at Micron's stated pin speed of more than 11 Gb/s the same width gives about 2.8 TB/s, matching its stated bandwidth.8 This is a computed illustration, not a figure from either company. The design choice is to run many signals at moderate speed rather than few at very high speed.
Stacking: dies, vias and bonding
Each stack is several DRAM dies on a base die, linked by TSVs. JEDEC's HBM4 supports 4-, 8-, 12- and 16-high stacks of 24 Gb or 32 Gb dies, for up to 64 GB in a 16-high stack of 32 Gb dies.2 Joining the layers is a manufacturing problem in its own right. SK hynix uses MR-MUF (mass reflow molded underfill), in which chips are stacked, a liquid protective material is injected between them and hardened; it says this dissipates heat better than laying film between each layer and helps control warpage.47 Samsung uses thermal compression with non-conductive film (TC NCF); in its 12-layer HBM3E it says thinner film let a 12-layer stack match the height of an 8-layer one, with a 7 µm gap between chips, and that bumps of different sizes are used, smaller for signals and larger where heat must escape.6
The base die and the interface
The base die sits under the DRAM layers and faces the processor. Samsung says its importance continues to grow with each generation, and builds its HBM4 on 1c DRAM with a 4 nm foundry process for the logic, with speeds of up to 13 Gb/s.9 JEDEC's July 2026 SPHBM4 standard shows what the interface die controls: it keeps HBM4's DRAM dies but adds a new base die that carries 512 data signals with 4:1 serialization instead of 2,048, so the stack can be mounted on a standard organic substrate rather than silicon, and a longer channel length from the processor becomes possible.3
Channels and reliability
JEDEC's HBM3 doubled the independent channels from 8 to 16, with two pseudo-channels each, and added on-die error correction with real-time error reporting.1 HBM4 doubles channels again to 32, adds Directed Refresh Management for row-hammer mitigation and defines vendor-specific supply voltages (VDDQ 0.7 to 0.9 V, VDDC 1.0 or 1.05 V), while keeping an interface that lets one controller work with both HBM3 and HBM4.2 More independent channels give designers more ways to access the stack at once.
Why it is hard to make in volume
Micron's 10-Q says HBM requires more wafers and more cleanroom space than conventional DRAM to produce the same number of bits at the same technology node, and names acceptable yields and quality for HBM products with their multiple chip layers among the technical barriers it faces.10 It also says it broke ground in January 2025 on an HBM advanced packaging facility in Singapore, expected to add capacity from the first half of calendar 2027, and is expanding DRAM and HBM capacity in Taiwan, with product shipments from an acquired fab expected from mid-calendar 2027.10 For what this has done to prices and supply, see The Memory Crunch, by the Numbers; the generation pages are HBM3E and HBM4.
Sources
- JEDEC, JEDEC Publishes HBM3 Update to High Bandwidth Memory (HBM) Standard (Arlington, Va., January 27, 2022), retrieved 2026-10-31 through the r.jina.ai reader, because jedec.org blocks direct requests.
- JEDEC, “JEDEC and Industry Leaders Collaborate to Release JESD270-4 HBM4 Standard” (April 16, 2025), full text as carried by Design & Reuse (original at jedec.org), retrieved 2026-10-31.
- JEDEC, New JEDEC SPHBM4 Standard Enables HBM4-Class Bandwidth on Organic Substrates (July 13, 2026), retrieved 2026-10-31 through the r.jina.ai reader.
- SK hynix Newsroom, SK hynix Begins Volume Production of Industry’s First HBM3E (March 19, 2024), retrieved 2026-10-31.
- Micron, HBM3E product page, retrieved 2026-10-31.
- Samsung Newsroom, Samsung Develops Industry-First 36GB HBM3E 12H DRAM (February 27, 2024), retrieved 2026-10-31.
- SK hynix Newsroom, SK hynix Completes World’s First HBM4 Development and Readies Mass Production (September 12, 2025), retrieved 2026-10-31.
- Micron, HBM4 product page, retrieved 2026-10-31.
- Samsung Semiconductor, HBM4 product page, retrieved 2026-10-31.
- Micron Technology, Form 10-Q for the quarter ended May 28, 2026, filed with the SEC June 25, 2026, retrieved 2026-10-31.