NVIDIA has provided a technical deep-dive of its Blackwell GB200 & GB300 racks, trays, and the Open Compute nature of its MGX platform.
NVIDIA Shares A Lot of What Goes On In The Development of Its GB200 & GB300 Blackwell Systems, Shows Open Compute Standards at Hot Chips 2025
Last year at Hot Chips, NVIDIA unveiled its first operational Blackwell servers, and this year, the company is rolling out its enhanced Blackwell Ultra platform. At Hot Chips 2025, the company presented a deep dive into its Blackwell rack/tray. The presentation, titled “Case Study: NVIDIA GB200/300, was presented by NVIDIA’s Mechanical Engineer, John Norton.
The conversation starts off with the introduction of the MGX architecture, which was contributed to OCP (Open Compute Platform) last year, and goes on to talk about some of the challenges involved in developing and designing the GB200 and GB300 “Blackwell” platforms.
MGX is an architecture that NVIDIA developed to address the challenges in regards to scaling accelerators executing across the world. It was found that it wasn’t just AI or inference, but other workloads that required a little bit of tweaking for specific customers. Some customers preferred a specific kind of management or just a NIC (Network Interface Controller, while some required a different mix of CPUs & GPUs, and what NVIDIA was doing was to continuously reiterate the system to change one small thing, but that one small change would ripple across the whole system. So that was one of the main challenges.
To address this challenge, NVIDIA came up with MGX. It’s a modular architecture. So they take the system and we cut it up into small pieces, building blocks, modules, and each of those can interoperate with a different set of modules, and they also established a kind of common set of specifications and interfaces for those modules to share. This allows NVIDIA to do an initial upfront investment on a single infrastructure, a single set of building blocks, and then only have to change one building block at a time as needed. By contributing this to OCP, MGX became open; it became something that customers could work within their supply base, and ultimately, allowing them to make those small customizations for their preferred implementation via the management, the NIC, etc, customers can design that into their own part of the system.
Next, John focused on two aspects of the architecture, the MGX rack infrastructure and MGX Compute & Switch Trays. These are the elements of that architecture that NVIDIA puts together to build GB200 “Blackwell”. NVIDIA uses an open set of standards in designing the MGX platform. It’s an open specification. NVIDIA publishes all the 3D models, 3D drawings in extremely long specs, and it’s all available for download today on OCP.
The company first discloses some high-level details of the Blackwell GB200/GB300 platforms. The picture on the left has the front view of the rack, while the picture on the right has the rear view of the rack. The top of the rack houses the switches. Directly underneath, there’s the rack power supply. What it does is it brings in the high-voltage AC from the data center and then converts it to DC output to the DC buck bar in the back of the rack.
Directly under the power supply, we have the compute trays. So in GB200, they have 300 chips within 10 compute trays, followed by nine switch trays, and then another eight compute trays. Each of those compute trays is capable of 80 FP4 Petaflops. The entire system, when we tie it all together with the switches, is capable of 1.4 exaflops. The entire system is approximately 120 kilowatts. Each compute tray is around 7 kilowatts. The whole system is tied together using the NVLink spine.
You can see that on the left-hand side and right-hand side of the back of the rack. So NVLink runs at 200 Gb/s per lane. It’s a low-latency copper interconnect, which also highlights the benefits of using copper. And so all of those GPU trays are tied together to switch trays using the NVLink spine.
So one of them was that the OCP rack specification deploys devices on a 48-millimeter pitch. Standard enterprise hardware that’s been in production for decades now has been deployed at a 44.5-millimeter pitch, approximately. And that’s what’s called a 1RU. Or it’s an EIA 19-inch standard. It’s got a lot of names, but they all mean the same thing. So what NVIDIA did is they took those EIA features and implemented them into the rack. So now they can deploy nodes at a tighter density, which has a lot of benefits.
There’s also the Bus bar. So the bus bar running up and down the back of the rack, which was capable of approximately 35 kilowatts, and NVIDIA needed about four times that much. So they created a deeper, larger bus bar, more cross-sectional area of copper to support up to 1,400 amps for the system. An extension frame was created that wraps around the back of the rack that taking it from a standard 1068-millimeter footprint to the next size up, the 1,200-millimeter footprint.
Next up, we have the NVIDIA GB200/300 NVL Compute tray. At a high level, each compute tray has two CPUs and four GPUs. There’s also an HPM, or the host-processor module. Each one of those supports one Grace CPU, two Blackwell GPUs. At the bottom of the block diagram is where all the IO connectivity comes out. And on the north side, there are NVLink connectors. Those are the connectors that are poking out of the back of the CPU. NVIDIA takes these HPMs and patches them up in what they call a micro-MGX board form factor.
The tray houses two HPMs, two board form factors. In the front, we have the cooling zone or the fan zone. This can be deployed in multiple configurations of fans, different channels, different ingress, and egress of cables. And then getting back to the modularity story, in front of the server, you can see three segments. This is where customers can develop their own management solution, develop their own NIC solution, deploy one set of SSDs or another set of SSDs, U.2 or E1S, DCSCM.
On the back of the tray, there are the UQDs. The UQDs are a universal quick disconnect. That’s also an OCP standard that NVIDIA leveraged. All of this is 100% liquid-cooled.
In the end, NVIDIA stated that both GB200 and GB300 are now in full production. They are scaling and already running in multiple hyperscale data centers across the world. The company is also following an annual cadence with which it will keep pushing the boundaries in terms of density, power, and cooling. Plus, technologies such as NVLink Fusion, an open interconnect solution, are really going to scale things up.
Source link
