- Akamai COO Adam Karon expects AI workloads to shift from GPUs toward CPUs as autonomous agents multiply
- Agents running on CPUs at the network edge can cut costs and latency, leaving GPUs to handle heavy inference
- Decision APIs and distributed infrastructure could open a substantial market for Intel, AMD, Arm and networking firms
The AI industry has spent billions building enormous data centers packed with GPUs. But what if the next computing revolution requires something rather different? More CPUs. More networking. More distribution. And considerably less dependence on the giant AI factories dominating investment and the industry's imagination.
That's the argument that emerged from my conversation with Adam Karon, COO and general manager of the Cloud Technology Group at Akamai, a company that knows something about solving the internet's architectural problems.
"There'll still be a place for GPUs, but I think the percentage will shift from GPUs to CPUs over time," Karon told me.
It's a remarkable prediction in an industry where GPU capacity has become synonymous with progress. And potentially good news for CPU suppliers including Intel, AMD and Arm's extensive ecosystem of chip designers, as well as the networking companies that will connect their machines.
The agents are coming
Karon isn't arguing that centralized data centers are obsolete. Large GPU clusters remain essential for training frontier models and other intensive workloads. The problem is what happens after those models leave the laboratory.
"The agents themselves become the new internet users," Karon explained.
These agents retrieve information, interact with enterprise systems, call APIs, execute code and make decisions. Every interaction introduces potential latency. Sending everything back to a distant GPU factory begins to look suspiciously like the mainframe era.
We've been here before. The internet escaped centralized computing's limitations by distributing content and processing closer to users. Akamai helped engineer that transformation through content delivery networks.
Karon said agents themselves can operate on CPUs at the network edge, while GPUs continue performing the inference workloads for which they're best suited.
The point isn't that CPUs replace GPUs. Intelligent systems need both, alongside storage, networking and security. The opportunity is to stop using expensive accelerated computing for tasks that don't require it.
The economics are compelling
Consider the difference in cloud computing costs.
An AWS instance equipped with a single Nvidia H100 GPU costs approximately $6.88 per hour on demand. A 64-vCPU AWS Graviton3 CPU instance costs approximately $2.32 per hour. The CPU instance costs about 66% less per hour. To be sure, that isn't an apples-to-apples comparison: the GPU instance is far more powerful for AI work. The gap measures the difference in hardware prices, not the savings on identical AI jobs.
For agent orchestration, API execution, business logic and other CPU-suitable workloads, the economics can be even more attractive when smaller instances are sufficient. Why allocate GPU resources to a task that a conventional processor can handle?
Of course, for highly parallel model inference, GPUs can deliver better performance per dollar. The opportunity isn't replacing GPUs indiscriminately. It's putting the right computing resource in the right place.
And when multiplied across millions of agents executing billions of interactions, the potential savings become substantial.
Shrek's Donkey
There's another issue: AI agents are extraordinarily talkative. Like Donkey in "Shrek," they don’t know when to shut up.
Each agent may communicate repeatedly with other agents, databases and applications. Multiply those conversations by millions of autonomous systems and the network implications become enormous.
The traditional telecom response would be to build a wider highway. But what if the answer is to reduce the distance those conversations travel?
"You want to make sure the systems that it's interacting with are close to that agent so you don't have to traverse the entire network the whole time," Karon said.
Akamai operates infrastructure across more than 4,000 edge locations globally. The opportunity is to place computing where an application needs it, rather than force the application to accommodate the computing's location.
Decision APIs: another opportunity for CPUs?
An emerging class of AI services makes Karon's argument particularly interesting: decision APIs.
Companies including TypeSafe AI, OpenAI, Microsoft, Cloudflare, Liquid AI and Perplexity are developing specialized decision models and interfaces that return choices, scores or probabilities rather than lengthy generated responses.
Instead of asking a large language model to reason through every small operational decision, applications can use specialized services to classify information, select actions, assess risk and determine what happens next.
This is a potentially important architectural change. An autonomous agent might make hundreds of small decisions while executing a single task. Sending every one of them to an expensive centralized model creates unnecessary cost and latency.
Not all decision models run on CPUs. Some still require accelerators, depending on their architecture and deployment. But the surrounding work — applying policies, managing state, routing requests, calling tools and executing decisions — is well suited to conventional processors. Smaller decision models may also be practical candidates for CPU-based execution.
And that creates an interesting possibility: distributing decision-making across the network rather than concentrating every operation in a handful of AI factories.
Akamai is already moving into this territory. Its agentic security framework includes a real-time decisioning layer for identity, trust and automated transactions at the edge.
The bigger opportunity is an emerging division of labor: GPUs for computationally intensive inference, CPUs for agent execution and orchestration, and specialized decision models for the enormous volume of smaller choices that autonomous systems will make.
For Intel, AMD, Arm's ecosystem and companies operating distributed infrastructure, that could become a substantial new market.
Where's the money?
AI's business case remains an uncomfortable question. Karon identified gaming, retail and broadcasting as areas where Akamai has seen AI applications move beyond pilots into production: responsive game characters, point-of-sale intelligence, transcoding and real-time dubbing.
But the obstacle isn't necessarily computing capacity.
"People try to layer AI on top of old systems and old data, and that doesn't work really well," Karon warned.
Enterprises must prepare their data and processes before optimizing the cost of each outcome. Otherwise, cheaper computing merely makes inefficient applications less expensive to run.
The sovereignty question
Distribution raises another question central to my coverage of our industry this year: who controls the infrastructure?
Sovereignty isn't simply about where data resides. It's about who can access it, move it or switch it off.
"[Enterprises] need to make sure that their data is portable and that their application's portable, so they're not locked in," Karon advised.
For companies deploying AI in mission-critical environments, that principle matters as much as latency. A distributed architecture can give enterprises more choices about where processing takes place, how information is protected and which providers they depend upon.
A different kind of AI infrastructure
Karon believes Akamai's distributed architecture provides some insulation from the debt, permitting and concentration risks of gigawatt-scale data center projects. Smaller facilities connected by a global backbone offer a different infrastructure model, although not one without investment risk.
Akamai evolved from distributing content to distributing security and now computing.
"This third wave that we're going through is really bringing compute out to that same location," Karon said.
The company is putting that strategy into practice with Akamai Inference and Agent Sandbox, combining GPU inference, CPU computing and secure agent execution across its distributed cloud.
The industry has spent years obsessing over how much computing power it can concentrate in one place. The next question is how intelligently it can distribute that power, how much of it actually needs a GPU and how cheaply the rest can be delivered.
For Intel, AMD, Arm's partners and the networking industry, that's a potentially enormous opportunity.
And for Akamai, it's a rather familiar business.
Learn more about AI infrastructure on Fierce Network
The data center correction is here. What happens next?
Ericsson's AI bet: The intelligent fabric vs the AI factory
T-Mobile's Ankur Kapoor: AI is moving from the cloud into the core network
Why telcos can't afford to ignore decision APIs
Stephen M. Saunders MBE is a communications analyst and USPTO-registered inventor examining how digital infrastructure — 5G, cloud and AI — is reshaping industry, power and society, as well as underpinning the emerging, ubiquitous global digital economy. As anchor of FNTV and a longtime industry insider, he focuses less on growth narratives and more on execution, risk and how hyperscale technology is distorting markets, governance and society at scale.
Opinion pieces from industry experts, analysts or our editorial staff do not represent the opinions of Fierce Network.