As NVIDIA continues to dominate the artificial intelligence (AI) market, major tech companies are ramping up their purchases of NVIDIA's graphics processing units (GPUs) while simultaneously accelerating the development of their own AI chips. This dual approach aims to enhance cost and energy efficiency by securing chips tailored to specific computational needs.
On August 25, OpenAI disclosed the initial performance metrics of its custom AI inference chip, named 'Jalapeño.' According to OpenAI, Jalapeño outperformed NVIDIA's GB200 and GB300 in both throughput per watt and response latency. Tests indicated that Jalapeño's throughput was 1.5 to 1.9 times higher than the comparison systems, while its response latency was 1.7 to 3.6 times lower. However, as these results are based on internal evaluations, the chip's competitiveness will need to be validated in commercial environments.
OpenAI plans to deploy Jalapeño in its computing infrastructure starting later this year, while continuing to utilize external accelerators, including NVIDIA's, for training and inference. This strategy does not aim to replace NVIDIA GPUs but rather to create a structure that employs multiple accelerators based on specific use cases.
OpenAI is not alone in its chip development efforts. Google has developed its own AI accelerator, TPU, for use in data centers. Amazon is creating its Trainium and Inferentia chips while also offering NVIDIA GPUs in its cloud services. Meta is expanding its AI chip, MTIA, while sourcing external chips from NVIDIA and AMD. Microsoft is also pursuing a strategy that combines its own AI accelerator, Maia, with external GPUs.
One of the primary motivations for developing custom chips is the rising cost of AI inference. As the number of AI service users grows, the computational demands for both training and inference have surged. While general-purpose GPUs can handle a variety of computations, they do not provide the same level of performance and energy efficiency for all tasks. Designing chips tailored to specific services can potentially reduce the power required for repetitive computations.
Inference is particularly seen as an area where the efficiency of custom chips can shine. While versatility is crucial for training, inference involves repeatedly executing pre-trained models, allowing for optimization tailored to specific models and services. This context helps explain why OpenAI developed Jalapeño as an inference chip.
The expansion of custom chips does not necessarily lead to a decrease in demand for NVIDIA GPUs. In fact, the overall demand for AI computations is rapidly increasing, resulting in a structure where big tech companies secure both external GPUs and their own chips. NVIDIA continues to collaborate with OpenAI, which is developing its own chips. In a recent earnings report, NVIDIA announced that its data center revenue surged 117% year-over-year to $89 billion.
However, the rise of custom chips could alter the competitive landscape of the AI semiconductor market. Traditionally, companies like NVIDIA and AMD have led chip design, with big tech purchasing these products. In the future, it is expected that big tech will increasingly design the structure and performance of chips themselves, collaborating with semiconductor manufacturers for production.
This shift may also change the customer structure within the semiconductor industry. High-performance semiconductors, such as high-bandwidth memory (HBM) and advanced packaging, are essential for custom AI chips. Reports indicate that OpenAI's Jalapeño utilizes HBM4. As more companies develop their own AI chips, there is growing anticipation that big tech's custom chips will become a new source of demand for high-performance memory, alongside traditional accelerator companies like NVIDIA and AMD.
* This article has been translated by AI.
Copyright ⓒ Aju Press All rights reserved.