Samsung is exploring memory chips that do AI processing themselves, not just store data for GPUs

[ShadowKernel]

Samsung is reportedly exploring a significant architectural shift in how high bandwidth memory works inside AI systems, moving beyond its traditional role as a fast data store that simply feeds GPUs and other accelerators toward a design where the memory itself handles memory control tasks and even performs selected AI computations directly. The approach represents a meaningful departure from how HBM has functioned throughout its entire history in AI hardware up to this point.

The underlying motivation is straightforward once you consider how modern AI systems actually spend their time. A significant share of total processing overhead in large AI workloads goes toward simply moving data back and forth between separate memory and compute chips, rather than the actual core calculations themselves. Performing certain operations directly inside the memory, particularly workloads involving especially heavy memory access patterns, could meaningfully cut down on that data movement overhead and improve overall system efficiency, since data that never needs to physically travel between separate chips never incurs that specific transfer cost or corresponding delay in the first place.

Thermal constraints remain the central engineering challenge holding this approach back from broader deployment right now. Packing more processing capability directly into an HBM base die inevitably generates additional heat in an already thermally constrained stack, which is exactly why Samsung is reportedly targeting selective, heavy memory access workloads specifically as the more practical and realistic starting point rather than attempting a wholesale redesign covering every possible AI workload at once.

This approach fits into Samsung's broader push toward what it calls 3D AI memory, most visibly demonstrated through a new architecture called zHBM unveiled at the FMS 2026 memory conference. Samsung claims zHBM could eventually deliver roughly eight times the performance of current generation HBM5 while also improving overall energy efficiency, a really ambitious target given how much raw performance the HBM roadmap has already delivered generation over generation in recent years. The move comes as Samsung works to reclaim ground in the broader HBM market from SK Hynix, which has held a clear competitive advantage through certification, production volume, and established key customer relationships specifically in the current HBM3E generation

Samsung Hbm Built In Memory Control Ai Processingwww.sammyfans.com

[Sharon79]

The thermal constraint problem described here is honestly the single most interesting engineering challenge buried in this whole piece. Adding processing capability directly into an already densely stacked, thermally constrained memory module is a fundamentally different and arguably harder problem than just adding more raw memory capacity or bandwidth the traditional way

[Anchor34]

Targeting selective heavy memory access workloads first, rather than attempting a full architectural overhaul covering every possible use case simultaneously, is a sensible, disciplined engineering approach. Solving the thermal and integration challenges for a narrower, well defined subset of workloads first before eventually attempting to broaden the approach out later makes considerably more sense than trying to boil the ocean all at once