HBM – Definition & Meaning

Definition of HBM

: a type of three-dimensionally stacked memory that uses a very wide connection to move data rapidly between memory and a nearby processor or AI accelerator, making it especially useful for training and serving large models
- The larger HBM capacity let the accelerator hold more model weights without moving them to slower host memory.
- Increasing compute alone did not improve inference speed because HBM bandwidth remained the bottleneck.

Origin & history

first known use2013

The first JEDEC High Bandwidth Memory standard, JESD235, was published in October 2013. HBM first appeared in commercial graphics hardware in 2015 and later became a core part of high-end AI accelerators as model size and memory traffic grew.

Test yourself

Which of these is the meaning of HBM?

Related terms

GPU graphics processing unit: a chip originally designed for video games that can do many… VRAM the memory built into a GPU; how much a model needs largely decides which graphics card can… KV cache memory that stores intermediate attention results for text a model has already processed, so… inference the act of running a trained model to produce an output, such as answering a prompt or… training the process of feeding data to a model and repeatedly adjusting its parameters so its outputs… data center a large building full of servers, chips, cooling, and power equipment; AI data centers can…

Cite this entry

"HBM." AI Dictionary, Dadgogo, https://dadgogo.com/dictionary/hbm/. Accessed 5 Oct. 2026.