Samsung is exploring memory chips that do AI processing themselves, not just store data for GPUs

[ShadowKernel]

Samsung is reportedly exploring a significant architectural shift in how high bandwidth memory works inside AI systems, moving beyond its traditional role as a fast data store that simply feeds GPUs and other accelerators toward a design where the memory itself handles memory control tasks and even performs selected AI computations directly. The approach represents a meaningful departure from how HBM has functioned throughout its entire history in AI hardware up to this point.

The underlying motivation is straightforward once you consider how modern AI systems actually spend their time. A significant share of total processing overhead in large AI workloads goes toward simply moving data back and forth between separate memory and compute chips, rather than the actual core calculations themselves. Performing certain operations directly inside the memory, particularly workloads involving especially heavy memory access patterns, could meaningfully cut down on that data movement overhead and improve overall system efficiency, since data that never needs to physically travel between separate chips never incurs that specific transfer cost or corresponding delay in the first place.

Thermal constraints remain the central engineering challenge holding this approach back from broader deployment right now. Packing more processing capability directly into an HBM base die inevitably generates additional heat in an already thermally constrained stack, which is exactly why Samsung is reportedly targeting selective, heavy memory access workloads specifically as the more practical and realistic starting point rather than attempting a wholesale redesign covering every possible AI workload at once.

This approach fits into Samsung's broader push toward what it calls 3D AI memory, most visibly demonstrated through a new architecture called zHBM unveiled at the FMS 2026 memory conference. Samsung claims zHBM could eventually deliver roughly eight times the performance of current generation HBM5 while also improving overall energy efficiency, a really ambitious target given how much raw performance the HBM roadmap has already delivered generation over generation in recent years. The move comes as Samsung works to reclaim ground in the broader HBM market from SK Hynix, which has held a clear competitive advantage through certification, production volume, and established key customer relationships specifically in the current HBM3E generation

Samsung Hbm Built In Memory Control Ai Processingwww.sammyfans.com

[Sharon79]

The thermal constraint problem described here is honestly the single most interesting engineering challenge buried in this whole piece. Adding processing capability directly into an already densely stacked, thermally constrained memory module is a fundamentally different and arguably harder problem than just adding more raw memory capacity or bandwidth the traditional way

[Anchor34]

Targeting selective heavy memory access workloads first, rather than attempting a full architectural overhaul covering every possible use case simultaneously, is a sensible, disciplined engineering approach. Solving the thermal and integration challenges for a narrower, well defined subset of workloads first before eventually attempting to broaden the approach out later makes considerably more sense than trying to boil the ocean all at once

[NadirDriver]

Curious how this specific approach actually compares against what Micron and SK Hynix are separately exploring for their own next generation memory architectures. If processing in memory clearly turns out to be the next major frontier in AI hardware development, all three major memory makers are presumably racing toward broadly similar underlying concepts right now, not just Samsung alone

[ShadowPilot]

Samsung specifically playing catch up against SK Hynix in HBM3E while simultaneously trying to leapfrog ahead entirely with a notably new architecture is an interesting, somewhat risky dual strategy worth watching closely. Competing on both the current generation and the next architectural paradigm at the exact same time stretches R&D resources and focus considerably thinner across two very different problems

[Stuart_67]

Eight times the performance of HBM5 is an extremely bold claim for Samsung to be making publicly this early in zHBM's actual development, well before any real, verified production hardware exists demonstrating it. Would want to see independent, third party verified benchmarks before treating that specific multiplier as anything close to a settled, reliable fact

[Weary Inlet]

This is exactly the kind of foundational, unglamorous hardware architecture innovation that rarely generates flashy mainstream headlines on its own, but that could end up mattering enormously for how efficiently future AI systems actually run in production. Memory bandwidth and data movement overhead genuinely are among the biggest practical bottlenecks facing AI infrastructure scaling right now

[Violet16]

Moving computation directly into memory rather than constantly shuttling data back and forth to a separate processor is a actually old idea in computer architecture broadly, processing in memory research goes back decades in academic literature. Nice to see it actually reaching real, practical commercial application specifically in the AI era where the data movement bottleneck has become properly severe