-
Companies are using CPUs, rather than GPUs, for some AI tasks
-
Enterprises are specifically using CPUs for AI inference jobs, an analyst told us
-
Nvidia is still on top for intensive AI workloads
Enterprises are often utilizing central processing units (CPUs) rather than the more recognized graphics processing units (GPUs) for artificial intelligence (AI) workloads.
The reason for that is simple and practical. Individual GPUs cost more than CPUs and a specialized GPU-based server can easily price out at hundreds of thousands of dollars.
So enterprises are often relying on CPUs for more generalized AI tasks, as J.Gold Associates analyst Jack Gold noted.
“This is defiantly happening. It’s actually a cost advantage for smaller training models, since most modern server-level chips have a fairly robust AI acceleration capability with built in GPU and even NPU capabilities,” Gold told Silverlinings in an email.
A Neural Processing Units (NPU), by the way, is a processor that specializes in the accelerated processing of machine learning algorithms, so there's another useless new industry acronym to learn!
“For inference processing, the general CPU/NPU based chips are dominant,” Gold said. Inference, you’ll recall, is the process of running live data through a trained AI model to make a prediction or solve a task. This is already used in retail to help best position the products to make a sale.
“If you look at the latest Intel Xeons, they have an AI boost capability with a built in NPU/accelerator, specifically targeted at these mid range training and general purpose inference operations. AMD is moving in the same direction,” Gold said.
AMD said that it has introduced its Ryzen 7040 Series processors for AI tasks. This chip first arrived in AMD-based x86 computers in this April this year.
Intel noted hat its 4th Gen Xeon processor family of products can support AI workloads.
“Nvidia still holds the high ground with its H100, but it’s a very expensive solution most attractive to the needs of very large models,” Gold said in an email. “But the Intel/AMD solutions means you can use existing servers, used for many other tasks, as AI enabled solutions as well.”
So maybe the cheaper solution will work for many enterprise AI tasks?