Mobile HBM: super fast dram for (Apple) smartphones with AI

The memory chip world market leader Samsung and probably also SK Hynix develop variants of low-power DDR (LPDDDR) SDRAM with extremely high data transfer rates. In addition, these have a lot more data lines than currently common LPDDR5X and upcoming LPDDR6 chips. Based on High Bandwidth Memory (HBM), which use the fastest AI accelerators from NVIDIA and AMD, for example, the new mobile store Mobile HBM could be called.

However, names such as Low Power Wide I/O (LPW) have also appeared. At the Samsung Memory Summit 2023 there was talk of low latency Wide I/O (LLW).

The underlying concept is not new. More than ten years ago, the industrial room published every standard for wide-i/o and Wide-i/O2 memory chips with up to 512 data signal lines. Wide-i/O-Dram was used, among other things, in the mobile game console PlayStation Vita.

Mobile HBM or LPW dram could be used after speculation from 2027, for example in iPhones and other smartphones with stronger AI data works. Until then, LPDDR6 dram should also be ready for market. This makes it difficult to assess the maximum data transfer rates of upcoming LPW stacking chips.

Data rating estimates

Current LPDDR5x-8500 chips with 16 data lines transfer 17 GB/S: 8.5 billion transfers with 2 bytes each per second. An LPW chip with a total of 256 or 512 signal lines (32 or 64 bytes), which consists internally from several stacked LPDDR5x-8500 dies, would therefore deliver up to 272 or 544 GB/s.

For comparison: A Apple M4 Pro with several LPDDR5X channels creates 273 GB/S, an NVIDIA GeForce RTX 5060 (TI) with GDDR7 memory it brings to 448 GB/S. High-end graphics cards come to much more than a TB/S.

LPDDDR5X is specified up to 9.6 gigatransfers/s (GT/s), then LPW would be possible with LPW over 600 GB/s. Samsung wants to drive “LPDDR5 Ultra Pro” to 12.7 GT/s.

LPDDR6 is said to start with 10.667 GT/s, with 512 data lines there are 682 GB/S. However, LPDDR6 chips should be organized in such a way that they process 24 instead of 16 bits per channel (2 sub-channels with 12 bit each). LPW based on LPDDR6 dies could therefore use 288 or 576 data lines.

Chip structure

For example, bond wires are used to electrically to connect the stacked this to the base carrier. In order to provide a large number of lines for very high signal frequencies, but vertically through the leading through contacts are better, so -called Through Silicon Via (TSVS). Several hundred TSVs fit on a square millimeter.

LPDDR dram dies with the side arranged on the side are easier and cheaper. If you then stack them in such a way that they collect slightly, each can be linked directly to the basic die (Redistribution Layer, RDL) by short vertical connections. SK Hynix has developed the technology Vertical fan-out (VFO).

Chip combination

In notebooks, LPDDR memory chips are typically soldered close to the mainboard next to the main processor. With LPCAMM/LPCAMM2 there is also a plug -in module version.

For mobile HBM, however, it should be necessary to stack the RAM package directly to the processor. This is the only way to keep the many lines short enough so that too many errors do not occur at high signal frequencies. Silicon interposers that connect CPU SoC and LPW dram side by side are also conceivable.

In the case of AI collectors for servers, the GPU and HBM stacks also sit on interposers. However, HBM uses 1024 data lines per stack and several stacks per GPU. Eight HBM3E stacks deliver around 8 TB/s.

Computing

Samsung and SK Hynix are also working on a specification with which processors can control arithmetic works that are integrated in RAM chips. With Processing-in-Memory (PIM), memory chips could deliver the results (simple) computing or search operations instead of just raw data. In principle, transfer performance and energy can be saved.