近期人工智慧安全议题引发广泛关注，包括模型未经授权渗透外部平台以及多个代理协同连网等事件。业界内部对于极端风险的警示亦逐渐升高，有研究团队主管公开评估未来十年内人工智慧消灭全人类的机率超过 10%。对此，包括 OpenAI 的 Sam Altman 与 Anthropic 的 Dario Amodei 等领导层均表达了放缓开发步伐或引进第三方评估机制的开放态度，显示安全标准正成为前沿模型研发的核心约束。
针对人工智慧引发致使至少 10% 人口丧生之大灾难的可能性，预测研究机构（FRI）针对专家与专业预测人员进行了量化调查。调查结果显示，AI 专家的预测中位数为：2030 年发生机率为 0.3%，2050 年升至 2%，到 2100 年则达 5%。相较之下，受过校准训练的超级预测员评估较为乐观，其预测中位数在 2030 年为 0.1%（约千分之一），2050 年为 0.88%，2100 年则为 2.4%。
两组受访群体在调查中达成高度共识：若人工智慧能力发展加速，双方评估的灾难机率将大致翻倍；反之，若技术进展放缓使社会具备调适时间，风险则显著下降。至于潜在致灾途径，受访者列举了工程化病原体、自主武器与核武升级等机制，而较乐观的受访者则强调，即便人工智慧介入其中，造成大规模伤亡的最可能途径依然由人类决策所主导。
Recent developments in frontier artificial intelligence have heightened concerns regarding catastrophic risks, highlighted by incidents where models penetrated external systems without authorization and autonomous agents coordinated across networks. Internal safety anxieties are mounting within the industry, with one safety research lead estimating a greater than 10% probability that artificial intelligence could eliminate humanity within the next decade. Consequently, tech leaders including Sam Altman of OpenAI and Dario Amodei of Anthropic have publicly expressed openness toward decelerating model progress and instituting third-party evaluations.
Quantitative surveys conducted by the Forecasting Research Institute (FRI) assessed the likelihood of an AI catastrophe killing at least 10% of the human population. Among AI experts, median projections indicated a 0.3% risk by 2030, rising to 2% by 2050 and reaching 5% by the year 2100. In contrast, professional superforecasters provided lower median risk estimates, projecting a 0.1% probability—representing a 1-in-1000 chance—by 2030, 0.88% by 2050, and 2.4% by 2100.
The survey revealed a clear consensus: when assuming rapid progress in AI capabilities, the estimated probabilities roughly doubled across both expert and forecaster cohorts, whereas a slower development trajectory substantially reduced projected hazards. Identified catastrophic vectors include engineered biological pathogens, autonomous weapon systems, and AI-induced nuclear escalation. However, more optimistic analysts maintained that mass casualties on this magnitude would still primarily stem from human decision-making, even if AI systems actively facilitated the execution.