Digital Realty's CTO said increasing rack weights have big implications for data center design
He noted cabling is also heavier simply because there's much more of it
The CTO also said he expects on-campus microreactors to become a reality in the not-too-distant future
It was practically a throwaway line. Mixed in with all the buzz about Nvidia’s new Blackwell GPU and DGX supercomputer, CEO Jensen Huang made an offhand joke about just how much its new powerhouse server rack weighs.
“Two years ago, when I saw a GPU it was the HGX. It was 70 pounds, 35,000 parts,” Huang said, explaining that he views GPU technology as encompassing the entire server rack. “Our GPUs now are 600,000 parts and 3,000 pounds. That’s kind of like the weight of a carbon fiber Ferrari.”
That last bit – the part about the new DGX rack weighing the same as a car – caught Digital Realty CTO Chris Sharp’s attention. Why? Because those racks will eventually end up in data centers, and the kind of weight Huang mentioned has huge implications for structural design choices, Sharp told Silverlinings.
Founded in 2004, Digital Realty has built a sprawling network of over 300 data centers in more than 25 countries across the globe. And plenty more are on the way.
In December 2023, it inked a $7 billion joint venture deal with private equity firm Blackstone to build four hyperscale data center campuses in Northern Virginia, Paris and Frankfurt which will eventually house a total of 10 data centers with 500 megawatts of capacity. Approximately 100 MW of that capacity will be online by next year. That means Sharp has a lot of work on his plate.
Sharp has been at the helm of Digital Realty’s technology operations for nearly nine years, having joined as CTO in mid-2015 after spending more than six years leading the charge on cloud services at Equinix.
He’s spent his off time equally immersed in technology. Sharp told Silverlinings that his family keeps chickens at his home in Colorado. But unlike most farmers, he ran a 10 gig optical link to his barn and built a neural network with his two children which uses computer vision to monitor the flock and alert them to intruders like raccoons.
Sharp is an AI enthusiast. But he can’t help but acknowledge that the technology – and the servers it requires – comes with a cost.
Heavier loads
According to the CTO, the growing weight, power, cooling and networking requirements of racks like Nvidia’s are changing the way data center facilities are built and retrofitted.
“It requires a thicker slab, more rebar just for the floor loading, and so that’s something you have to anticipate,” he said of the increased rack weight. “It’s much easier to do heavier floor loading with slab on grade [rather than raised floors], and so we’ve been engineering with more rebar and just making sure that can be supported.”
For a multi-story facility, though, heavier rack weights translate to more cost and complexity, he said.
“I think we’re hopefully reaching the end of the weight, but we’ll see. There’s only so far densification can go,” Sharp said.
Crazy cabling
Besides the racks themselves, Sharp said an increase in the amount of cabling required to support the servers is also adding weight.
“It’s like five to ten times more cables within the four walls of the data center,” Sharp said. As a result, he said he spends a fair bit of time looking at – of all things – cable trays to ensure they can handle the weight of the cabling required.
Sharp isn’t alone in pointing this out. Corning Optical CTO Mike O’Day told Silverlinings last year artificial intelligence would likely require at least five times more optical connections in the data center. Luke Kipfer, VP of Data Center Development and Construction for PowerHouse Data Centers, recently reiterated the fact that an increasing amount of space is needed to run fiber within data centers to support high-power racks.
And don’t even get us started on the whole Ethernet vs Infiniband debate. Actually, do, because there’s something we need to address.
Bandwidth arguments aside, Sharp noted Infiniband comes with one key handicap – it has a distance limit of about 50 meters. That has huge implications for what can go where and how many GPU cores can be connected inside each facility, he explained. Simply put, Ethernet doesn’t have the same distance limitation. Take from that what you will.
But the challenge of solving puzzles like these – moving the pieces around until they fit – is right up Sharp’s alley. He noted that during his time at Equinix, the company built an Ethernet exchange with cloud interconnects. When it was initially brought to market it didn’t get any traction and “failed miserably.”
He noted, however, the same technology was subsequently repurposed for Equinix’s Cloud Exchange, which is now a hugely successful business. That whole process of ideation taught him a valuable lesson. Sometimes you have the right pieces in your hand, but they need to be turned just a little to fit the puzzle.
“It’s all about perspective. Looking at it one way, it doesn’t work. But then wow. It took off” with just a few tweaks.
Nuking the power problem
According to Sharp, there are a number of other puzzle pieces data center operators like Digital Realty are juggling.
Beyond server weight and cabling, they’re also in an “arms race” to make liquid cooling solutions available as quickly as possible. Thus, he said, Digital Realty is “pre-procuring and stockpiling” both rear door heat exchanger and direct-to-chip cooling systems alike.
Data center builders are also trying to piece together new solutions to meet rising power demands for their facilities.
In many areas, grid-based power supplies are being stretched thin. We recently noted that hyperscalers are starting to eye nuclear power as an alternative energy source and Sharp said the idea isn’t at all far-fetched.
“I’m a firm believer that small modular reactors will be needed to fortify the grid,” Sharp said.
He specifically highlighted a company called Oklo which is backed by OpenAI CEO Sam Altman. (For what it’s worth, Altman has also invested in nuclear fusion company Helion Energy.) Oklo is working to bring modular fission reactors to market. Its website states it is targeting its first commercial deployment in the 2026 or 2027 timeframe.
“If you don’t have that power, it’s absolutely going to be a constraint, and so that’s something we’ve been watching for some time now,” Sharp said. He added it is “highly likely” modular reactors will be built on data center campuses in the future and stated “you’ll see us become more public over the years on exactly how that’s going to take place.”
How many years? Sharp couldn’t provide an exact figure but said “two or three.”
“I think you’re going to see in the next year more details around the test case that they’ve built and how it’s performed and then a year after that we’re going to start to see this being very viable,” he concluded.