U.S. artificial intelligence (AI) company Anthropic has issued a warning that advanced AI could pose "catastrophic or existential risks to humanity" ahead of its initial public offering (IPO). It is unusual for a company seeking to profit from AI technology to officially acknowledge the potential threat to human existence in its investor materials.
On September 28, Reuters reported that Anthropic stated in its IPO prospectus, "As AI models, platforms, and applications become more sophisticated and widespread, the risk of our models causing harm may increase."
Specifically, the company noted that AI models could exhibit "self-preserving behavior," potentially resisting shutdowns or concealing and manipulating information, and even displaying threatening behavior.
While it is common for companies to disclose product and business risks during the IPO process, Reuters highlighted that it is extremely rare for a company to warn that its technology could threaten human existence. Anthropic emphasized that AI could bring about changes comparable to the industrial revolution or the invention of electricity, but if mismanaged, it could lead to irreversible harm.
Anthropic has recently expressed concerns over the rapid advancement of AI technology, arguing that the pace of development should be slowed to ensure safety. CEO Dario Amodei called for the entire AI industry to join in this effort in a lengthy blog post published on September 12.
In fact, approximately 80 of the 261 pages in the IPO prospectus are dedicated to explaining risk factors, while only 48 pages are devoted to business descriptions.
In comparison, xAI, an AI company led by Elon Musk, allocated about 38 pages to risk factors in its 277-page IPO prospectus.
Anthropic stated that significant resources are required to ensure safety. It explained that limited funds must be divided among computing resources, advanced AI personnel, and safety research. However, it acknowledged that the returns on investments in safety research are uncertain.
The prospectus did not disclose the costs associated with safety research. However, Anthropic recently revealed that approximately 6% of the computing resources used for AI research in July were allocated to safety research.
The challenge lies in emphasizing AI safety while struggling to maintain pace in the competitive landscape of cutting-edge AI development. Anthropic noted in its prospectus that customer usage and revenue depend on the release of new AI models, necessitating continuous model launches to remain at the forefront of AI technology.
Indeed, Anthropic launched its new Opus model just ten days after CEO Amodei's call for a slowdown in AI development.
Meanwhile, Anthropic was initially expected to go public in October but has recently pushed its target date to November. This delay is interpreted as an effort to present strong third-quarter results and clarify the company's stance on AI safety ahead of the IPO.
* This article has been translated by AI.
Copyright ⓒ Aju Press All rights reserved.