I attended the 2026 IEEE Hot Chips Conference at Stanford University this week. There was a record audience at the event, several hundred people, maybe 1,000. Likewise at the recent FMS Conference in Santa Clara there were more attendees than any FMS since the pandemic. The size of the audiences is an indication about how hot hardware has become.

This isn’t to say that software isn’t important. HW and SW are both important to enable technology innovation, but the current pace of AI innovation has put a big emphasis on developing new hardware and this is particularly true of memory and storage technology where the data to run AI workloads lives. Let’s look at some of the innovations in memory and storage from the 2026 Hot Chips Conference.

On Sunday, Jim Handy kicked off the tutorial sessions with a talk about how memory is feeding AI’s voracious hunger for data. This was followed by talks by Micron Fellow Raghu Sreeramaneni and Sangwook Han, Design Lead for customer HBM4E at Samsung about HBM current and future developments.

oth talked about off-loading some compute functions into the base die of an HBM stack. The figure below is from the Micron talk showing expected trends in HBM cube bandwidth, energy efficiency and memory capacity.

Major challenges facing higher DRAM die stacks include coefficient of thermal expansion, CTE, mismatches between various materials in a heterogeneous die stack and thermal issues due to the energy required to refresh the DRAM die in an HBM stack. Various die bonding and packing approaches are also required, including Cu-Cu hybrid bonding.

The Samsung tutorial went into great depth on future advances in HBM, including the company’s zHBM where the HBM is directly mounted upon a processor rather than next to the processor, as in modern GPU packages.

Samsung identified three phases of HBM base die, the die that all the DRAM dies are stacked upon, opportunities. Phase 1 involves shrinking logic processes on the base die to reduce its power consumption and to allow new logic functions on this die. The part of the die used to support die to die communication can shrink, allowing die space for more compute functions on the base die. He also described a heat bath block that ran along the side of the DRAM die in the stack that would help keep the DRAM die cooler. He also discussed using unused B-die space with the finer features for SRAM repair resources.

The second phase exploits the additional available die space with shrunken logic features for add additional functions to the HBM stack using the base die. They call this stack a cHBM compared to conventional sHBM. This might enable features such as direct memory extension from the B-die to connect directly with external memory. It could also enable offloading some of the XPU computation to processing elements, PE, in the base die and moving data to these Pes without going through the XPU.

The third phase is zHBM, where the memory stack is placed directly on the XPU using an interlayer die. As the figure below shows, this provides shorter data paths between memory and processor than having HBM stacks besides the XPU. Besides faster performance, zHBM could improve power efficiency by 70%

Sudeep Ghoja and Aayush Ankit from D-Matrix gave a tutorial of what they call 3D DRAM, where, instead of putting logic beneigh the DRAM die stack they want to put the logic on top of a DRAM die stack as shown below.

D-Matrix says that can get higher bandwidth for lower data movement energy with their device than HBM4. They also focused on a 4-high DRAM stack rather than the 12-16 high stack with current HBM. The result is bandwith with a DRAM stack that approaches SRAM with much higher capacity than SRAM. The company was showing their Raptor 3D-DRAM accelerator at the Hot Chips Conference that had a TSMC N4 logic on top of a DRAM die.

There were many other storage and memory related talks at the Hot Chips Conference and we will see if we can get back to talk about some of the other presentations in the future.

The 2026 Hot Chips Conference showed ways to expand HBM technology, including Samsung’s zHBM as well as other AI memory devices such as D-Matrix’s 3D DRAM.