- Network operators and vendors are investing in data center and network infrastructure to meet AI demand
- AI adoption will require edge investments
- Upgrading versus starting from scratch is a key question
Network operators and vendors are making big investments for the AI boom, including upgrading old infrastructure and building new.
Currently, most AI-driven infrastructure investments are focused on hyperscalers training large models that require tens or hundreds of thousands of GPUs and accelerators, said Dell’Oro analyst Sameh Boujelbene. These GPUs must be connected in large clusters through a scale-out infrastructure. Data center switches for AI back-end networks will drive nearly $80 billion in spending over the next five years, Dell’Oro predicts.
Enterprises and telcos are still in the early stages of AI, but Dell’Oro expects their adoption to accelerate, with most investments focused on the edge.
Organizations will likely upgrade existing data center networks on the front end to add new AI inferencing solutions into existing services, said Brendan Gibbs, VP of AI, Routing and Switching Platforms at Arista. Spending on ultra-scale back-end network infrastructure for language model (LLM) training will inevitably lead to complementary investments to build front-end networks for AI inference, he said.
This combination of back-end and front-end networks for AI represents “the evolution of traditional data centers into the new era of the AI Center,” he told Fierce.
Upgrading the old and moving to the edge
While starting over and building brand new data centers with bigger footprints “might seem ideal, that isn’t the approach most of our customers are taking,” said Cisco’s Murali Gandluru, VP of product management for Data Center Networking.
AI is following a “similar path” to the cloud shift, but with a much higher barrier to entry for overhauling infrastructure to meet the demands of specific AI use cases, Gandluru said. Most enterprises still don’t know their exact AI use cases, but optimizing infrastructure now will position them “to adapt faster as AI demands evolve and become more defined.”
Necessary data center upgrades will require rethinking how data is stored, processed and accessed to keep pace with the evolving landscape. As enterprises ready to deploy and scale their AI strategies, “they are looking at their current footprint and network infrastructure and identifying areas where they can refresh their infrastructure to be AI-ready,” he said — for example, deploying 400G/800G switching to enable the data center to support AI training and applications.
Most telcos are already exploring AI and generative AI (GenAI) to drive operational efficiencies and employee productivity. As GenAI matures, it will be used to support the user experience better, said Dan DeBacker, SVP of product management at Extreme Networks. “This will likely cause a need for infrastructure upgrades to accommodate the applications currently in use simply,” he told Fierce.
Extreme’s CIO Insights Survey showed that 49% of responding tech leaders experienced bandwidth challenges when they began implementing AI. Not surprisingly, spending is increasing to meet these challenges head-on, with 86% of respondents planning to invest in their networks over the next 18 months and 38% prioritizing efforts to improve network performance.
“The chances are high that most businesses are approaching these bandwidth concerns from the data center and deploying their workloads in that direction,” DeBacker said. With the move from centralized AI to deconstructed and distributed AI, IT professionals must reevaluate their enterprise network strategy. More network bandwidth will need to be pushed toward the edge to IoT devices and this distributed model.
Building new for AI
On the other hand, significant investments have gone into building completely new infrastructure for AI. The industry has witnessed the rise of the “AI factory,” a.k.a new data centers purpose-built for AI applications and power consumption.
To cater to the kind of AI demand we are likely to see, “there is a clear business case” for new infrastructures that are AI-native, said Ronnie Vasishta, SVP of telecom at NVIDIA. That means building new infrastructure designed for AI, “making them a future-proof investment, rather than retrofitted for AI, which would not scale into the future,” he told Fierce.
Legacy network systems will not be able to support AI unless three things happen, according to Vasishta:
- Underlying computing infrastructures evolve from single-purpose to multi-purpose systems. For example, the same infrastructure can be used for multiple applications like radio, AI, core functions, and orchestration, versus building siloed bespoke infrastructure for each type of application or function
- Systems are capable of doing accelerated computing
- Workloads are fully software-defined, allowing them to run on any multi purpose infrastructure
Telcos are already investing in accelerated computing as the foundation of new infrastructure, with RAN as one of many workloads. This accelerated computing foundation, designed and optimized for AI, will “pave the way for new telco revenues as tokens providers," Vasishta said.