- Red Hat is scooping up AI optimization startup Neural Magic
- It also unveiled a slew of incremental updates aimed at facilitating enterprise AI lifecycle management
- Both are part of Red Hat's goal to carve out an AI niche at the enterprise edge
Red Hat is going on offense, sprinting to an open corner of the AI field with a fresh acquisition and incremental platform updates that put it in perfect position to take a pass from enterprises looking to run inferencing workloads at the edge.
Its headline play is the acquisition of Neural Magic, an AI startup known for using optimization and compression to shrink AI models with minimal loss to accuracy. What this does in practice is allow enterprises to run complex AI workloads which normally require a top-tier Nvidia GPU on commodity processors (think CPUs from ARM or Intel). Neural Magic has notably raised $50 million in funding from a variety of backers and applied its technology for Meta’s Llama 3.1 model.
AvidThink Founder Roy Chua noted the deal reflects an industry shift away from raw performance in favor of optimization for specific use cases. See also: Nvidia’s acquisition of Run:ai and Intel’s SigOpt buy.
But Chua added Neural Magic also has an open-source ethos that aligns well with Red Hat’s approach to Kubernetes. The company is an active contributor to and user of the vLLM open source library. Given Red Hat wants to recreate its success with Kubernetes in the cloud with something in the AI realm, well…then as Chua put it, the deal “represents a significant move in the open-source AI space, potentially creating a more standardized approach to AI model deployment and optimization (leveraging the open-source vLLM inference runtime) across different hardware architectures.”
Incremental upgrades, big ambitions
However, the Neural Magic news is just one piece of the puzzle Red Hat is putting together.
Red Hat this week also unveiled a series of updates for its OpenShift, OpenShift AI and Device Edge platforms. These include data drift and bias detection in OpenShift AI, confidential compute attestation in OpenShift, and low latency features for Device Edge.
While incremental, Chua said these updates show that Red Hat is “serious” about supporting enterprises as they move through the lifecycle of building, managing and maintaining AI.
“Red Hat thinks that there is still an opportunity at the edge, and they want to invest in enterprise edge because that’s away from the cloud, so strategically the believe they have value there. And so now the intention is to make it easy for you to build models, test models, manage models and deploy models both in the cloud and at the edge especially,” he said. “If you look at that direction and that strategy, the Neural Magic acquisition absolutely makes sense.”
Indeed, neXt Curve Founder Leonard Lee told Fierce that “Device Edge is interesting in that it extends Red Hat’s OpenShift footprint out to endpoint devices residing and transversing enterprise edge environments. This extension of the Red Hat portfolio positions the company to cover the hybrid AI continuum from cloud to device.”
Lee added that on the tooling front, it seems Red Hat could also be filling a key gap in the market. Features like the ones Red Hat is bringing to market are being “introduced as enterprise POCs and pilots expose the limitations, cost, and full lifecycle support needs of GenAI systems and a growing range of compound GenAI paradigms and architectures such as RAG (Retrieval-augmented Generation),” he said.