Leaks reveal beefed-up specifications for Nvidia’s next-gen GB300 AI server

Details of Nvidia Corp. The next-generation AI server, the GB300 platform, has been leaked by Chinese media and is expected to bring big improvements in memory, performance, connectivity and cooling.

The world’s most valuable chipmaker is expected to announce the GB300 AI server at its annual GTC event, scheduled for March 2025, but while it was hoping to keep the new platform under wraps until then, it appears it won’t. failed to do so. .

The Chinese-language media site UDN complaints as supply chain sources revealed detailed specifications about the new server. The GB300, which will be powered by Nvidia’s most advanced Blackwell graphics processing units, is said to benefit from a significant memory increase, with 288GB of HBM3e RAM, compared to just 192GB on its predecessor, the GB200 AI server.

Additionally, Nvidia has changed the architecture from eight layers to 12, while the computer motherboard now uses a low-power compression attached memory module.

The Nvidia B300 chips at the heart of the GB300 will require 1,400 watts of power, and network speeds have been improved with the upgrade from ConnectX 7 to ConnectX 8, increasing bandwidth from 800 gigabytes per second to 1.6 terabytes per second.

The report also mentions a 50% increase in performance of the FP4 compared to the GB200. The decision to use FP4, which means four bits of floating point precision per operation, is said to be one of the main reasons people are so excited about the GB300. The reduced precision achieved by upgrading to FP4 results in faster computation, reduced data movement, and reduced power consumption, making it better suited for AI inference workloads.

There are other upgrades as well, with the use of a new set of “slot design” that will be introduced in the GB300 server and a new capacitor tray.

UDN claims that the upgrades mean that the performance of the GB300 server will improve on that of the GB200 “in every aspect”. He adds that the server is emerging as Nvidia’s “next weapon to conquer the market”.

It’s no surprise, however, that these improved specs come at a considerable cost, and the GB300 will likely cost a staggering price. According to UDN, its supply chain sources estimate that the total cost of producing a single GB300 supercapacitor will be between $20,000 and $25,000. With the GB300’s NVL72 AI server cabinet requiring more than 300 of these supercapacitors, customers such as Amazon Web Services Inc., Microsoft Corp. and Google LLC can expect to pay at least $7.5 million to complete one.

What’s unclear is when the GB300 will enter mass production. Its predecessor, the GB200, is still ramping up and deliveries are not expected to peak until the middle of next year, following delays due to late design flaws in Blackwell GPUs that reportedly caused overheating problems.

These setbacks mean Nvidia has an order backlog of about a year. Indeed, the AI ​​boom has led to intense demand for computing capacity and Nvidia’s GPUs are generally considered the best in the industry. Thus, Nvidia’s market capitalization rose to more than $3 trillion, making it one of the three most valuable companies in the world.

Although the focus so far has been on enterprise, Nvidia is also about to roll out its first consumer Blackwell GPUs, and insiders say we could hear more about it at the Consumer Electronics Show in next month.

Image: SiliciumANGLE/Meta AI