Raja’s new role will be to guide Sandisk into the development of HBF or High Bandwidth Flash memory for AI intensive GPUs, since the traditional HBM memory brings bottlenecks when it comes to capacity.
Raja Koduri Joins the HBF Memory Advisory Board at Sandisk to Advance the Goal of Delivering up to 4 TB Memory Capacity on AI GPUs
After retiring from Intel’s Graphics department as the Chief Architect in 2023, Raja Koduri is now on his new mission to accomplish higher VRAM capacity on AI GPUs. Raja announced that it is now joining the Technical Advisory Board at Sandisk, a popular storage device maker, which will now also focus on developing the HBF (High Bandwidth Flash) memory to deliver a large capacity memory solution for AI GPUs.
Raja’s collaboration with Sandisk is a significant move due to his background in the development of GPUs, particularly his experience in designing compute architectures, which perfectly aligns with Sandisk’s mission to produce the HBF for achieving higher capacity VRAM. Sandisk has announced the formation of the Technical Advisory Team in a press release, sharing Raja Koduri’s vision to eliminate the bottlenecks HBM (High Bandwidth Memory) suffers from.
When we began HBM development our focus was improving bandwidth/watt and bandwidth/mm^2 (both important constraints for mobile), while maintaining competitive capacity with the incumbent solutions. With HBF the focus is to increase memory capacity (per-$, per-watt and per-mm^2) significantly while delivering competitive bandwidth.
– Raja Koduri
While HBM has advanced quickly, delivering high memory capacities for AI superchips, the HBF can drastically increase memory capacity by leveraging the Through-Silicon Vias technology. With a single HBF stack, it can achieve terabytes of memory capacity, and integrating eight of such stacks in a system can help the AI GPUs reach up to 4 TB of VRAM, while retaining the high bandwidth HBM brings to the table. This helps overcome the limits of HBM, which isn’t able to catch up to the intensive AI demands and needs much more time to advance.
It’s important to note that Sandisk HBF won’t be competing directly with the DRAM in latency-critical workloads, since AI operations such as AI inference and large-scale model developments have a higher need for more memory capacity to deliver quick results. Higher capacity and bandwidth outweigh the raw latency requirements in such operations, and Raja’s role will be a strategic one in developing such a high-capacity HBF.
Since Sandisk is pushing to establish HBF as an open-standard ecosystem, it will encourage a broad industry adoption. Raja’s extensive network and experience in ecosystem-building will be excellent for this task, which will further promote partnerships with GPU vendors.
HBF is set to revolutionize edge AI by equipping devices with memory capacity and bandwidth capabilities that will support sophisticated models running locally in real time,”
This advancement will unlock a new era of intelligent edge applications, fundamentally changing how and where AI inference is performed.
– Raja Koduri
News Source: SanDisk
