On Wednesday, OpenAI unveiled its first custom-built inference processor, designed and manufactured in collaboration with Broadcom. Named Jalapeño, the brand new processor was designed particularly for the distinctive wants of OpenAI’s inference programs. OpenAI’s personal AI fashions assisted within the improvement of the chip, the corporate mentioned.
Whereas the chip remains to be being examined, OpenAI says early outcomes present considerably higher performance-per-watt than present state-of-the-art options.
The partnership was formally introduced in October, however OpenAI’s chip plans have lengthy been rumored as a approach to cut back the corporate’s dependence on Nvidia’s GPUs. Google and Amazon have each constructed {custom} chips to serve an analogous function, typically referred to as “AI accelerators” — silicon designed particularly to hurry up machine studying workloads.
OpenAI president Greg Brockman defined the corporate’s strategy to chip improvement on its in-house podcast, shortly after the Broadcom partnership was introduced.
“We’ve got a deep understanding of the workload,” Brockman mentioned within the episode. “We’ve actually been searching for particular workloads which are underserved, [and asking] how can we construct one thing that may be capable to speed up what’s attainable?”
Jalapeño is particularly designed for inference, the method of operating pre-built AI fashions in response to consumer instructions. Within the announcement, OpenAI emphasised the chip’s low working value when operating real-time coding fashions. It’s probably that extra performance-intensive duties like pre-training will nonetheless depend on Nvidia {hardware}, however even small reductions in inference prices may do rather a lot to enhance the corporate’s backside line.
Optimizing that inference system might show to be a vital issue within the economics of AI going ahead — and it’s more likely to happen at each degree of the stack. OpenAI is already constructing agentic merchandise like Codex and the fashions that energy them, in addition to knowledge facilities to run these fashions. Transferring into purpose-built chips lets the corporate go even additional in that course of, as the corporate defined in its announcement.
“OpenAI isn’t solely growing frontier fashions or constructing merchandise on prime of them; it’s designing the infrastructure beneath them: chip structure, kernels, reminiscence programs, networking, scheduling, deployment programs, and product expertise,” the corporate wrote. “As a result of OpenAI operates throughout the stack, every layer might be optimized across the identical objective: making its fashions quicker, extra dependable, and extra inexpensive for customers.”
If you buy by means of hyperlinks in our articles, we might earn a small fee. This doesn’t have an effect on our editorial independence.
