来自 Anthropic、OpenAI、Meta 和微软等顶尖人工智慧企业的高层与学者在一篇新论文中呼吁政策制定者加强审查 AI 的自我改进程度。他们对利用 AI 自动化开发未来模型的趋势表示担忧，认为这可能触发技术进展迅速加速的「智慧爆炸」，进而使技术能力超出社会引导和适应的范围，并削弱人类对研发过程的监督。

研究人员敦促政策制定者深入掌握顶尖实验室中 AI 自动化研究的运作情况，并制定机制来引导与限制智慧爆炸，包括对涉及自动化研发的资料中心实施更严格的安全措施与监管，同时制定应对各类 AI 爆发场景的紧急应对计划。Anthropic 本月亦透露，其超过四分之一的 AI 研发工作已由其 Claude 聊天机器人主导，显示软体驱动智慧爆炸的潜在可能性。

面对未来先进 AI 系统可能带来的灾难性风险，业界领袖已升级示警力道。Anthropic 执行长 Dario Amodei 呼吁放缓最先进模型的开发节奏，并获得了 OpenAI 的 Sam Altman 和 Elon Musk 的支持；Amodei 与 Altman 亦在联合国安理会上呼吁各国领袖携手合作。专家警告，相较于面临的高风险，人类目前的准备显然不足，一旦智慧爆炸开始，采取行动的机会之窗恐将关闭。

Top executives and researchers from leading AI organizations, including Anthropic, OpenAI, Meta, and Microsoft, published a paper urging policymakers to scrutinize AI self-improvement. They raised concerns that automating the development of future models could trigger a rapid acceleration termed an 'intelligence explosion,' which poses risks of AI capabilities outpacing societal governance and diminishing human oversight in research and development.

The authors called for policymakers to gain greater transparency into automated AI research across leading labs and develop frameworks to steer and constrain an intelligence explosion. Recommended steps include stricter safety standards and oversight for data centers handling automated AI R&D, alongside emergency contingency plans for rapid AI takeoff scenarios, especially as Anthropic revealed that over a quarter of its AI R&D is already led by Claude.

As concerns over catastrophic risks from advanced AI escalate, industry leaders have stepped up warnings. Anthropic CEO Dario Amodei publicly advocated for slowing the pace of developing the most powerful frontier models, garnering backing from OpenAI's Sam Altman and Elon Musk, as well as joint appeals to the UN Security Council. The researchers cautioned that humanity remains insufficiently prepared and that the window to act could rapidly close once an intelligence explosion begins.