- Data center chips still talk to each other through electrical technology
- But IBM’s new technology allows chips to talk via the power of fiber optics
- This could speed chip-to-chip communication 80x
IBM says it’s created a breakthrough in optics technology that could dramatically improve how data centers train and run generative AI models. Its new co-packaged optics tech basically brings the power of fiber optics onto a chip to enable connectivity within data centers at the speed of light.
Speaking to reporters at a media briefing, Mukesh Khare, general manager of IBM semiconductors, said the telecommunications industry has made significant progress in building faster and faster chips. But the speed at which these chips talk to each other has not grown as fast. There is a gap of several orders of magnitude between the rate at which compute has grown and the rate at which communication between chips has kept up.
He said, “Essentially the chips at the more fundamental level still communicate electrically. They use copper wire. And as we all know, our best communication technology is fiber optics, and that's why fiber optics is used everywhere else for long-distance communication.”
Although co-packaged optics tech has been around for a while, IBM has created new polymer optical waveguide (PWG) technology to power the co-packaged optics. PWG enables chipmakers to add six times more optical fibers at the edge of a silicon photonic chip compared to what was possible before on this “beachfront property.” Each fiber, about three times the width of a human hair, could span centimeters to hundreds of meters in length and transmit terabits of data per second.
What does all this mean?
The company said the technology translates into 80 times faster bandwidth than today’s chip-to-chip communication that uses electrical technology, and it will reduce energy consumption by more than five times.
It can also train a large language model (LLM) up to five times faster, reducing the time it takes to train a standard LLM from three months to three weeks, with performance gains increasing by using larger models and more GPUs.
In addition to allowing GPUs and accelerators to talk to each other much faster, it could also redefine the way the computing industry transmits high-bandwidth data through circuit boards and servers.
“We are very excited that we are bringing the power of light to essentially accelerate the world of Gen AI and many other applications,” said Khare.
Asked when the technology will be commercialized, Khare said IBM’s research department is “ready with respect to usage.”