在一名离职员工公开警告人工智慧可能威胁人类生存后,Anthropic 执行长达里奥·阿莫代(Dario Amodei)发表文章提议 AI 实验室放缓尖端研究步伐,并建议由外部稽核员进驻检查各家公司的安全规范。然而,这项提议被批评流于表面且不够彻底,因为若技术真有灭绝人类的风险,仅仅呼吁放慢研究或引入外部稽核并无法真正消除隐患,更应该彻底停止最具风险的递归自我改进(RSI)等尖端技术开发。
尽管全面中止尖端 AI 研究可能冲击相关公司的商业利益与公开上市计划,但历史上不乏为了公众安全而主动暂停研究的前例,例如生物学家曾在 1970 年代主动暂停高风险基因拼接实验。与此同时,当前企业市场更迫切的需求是将既有、成本较低且实用的人工智慧模型整合到现有工作流程中,AI 公司若转向扎实的商业落地与应用服务,即便不盲目角逐超级智慧,依然能创造可观的商业价值。
阿莫代与矽谷主流先驱陷入了一种矛盾心态:他们一方面承认对模型内部运作机制的理解仍非常有限,甚至担心具备欺骗与入侵能力的自主系统可能失控威胁网路安全,另一方面却仍坚持认为超级智慧的出现无可避免,必须抢先开发出乌托邦式的成果。作者指出,「技术必然性」绝不能成为继续进行可能毁灭人类文明之研究的借口,阿莫代所谓的放慢脚步应当转变为彻底终止超级智慧的竞逐。
Following public warnings from a former employee about artificial intelligence posing an existential threat to humanity, Anthropic CEO Dario Amodei published an essay proposing that leading AI labs decelerate their cutting-edge research and allow external auditors to monitor safety practices. However, this proposal has faced criticism for being vague and inadequate, as merely slowing down or introducing audits fails to address the core peril; if the technology genuinely threatens human survival, research into high-risk areas like recursive self-improvement should be completely halted.
Although pausing frontier AI development might disrupt financial prospects and initial public offerings for AI pioneers, scientific history offers precedent, such as biologists voluntarily placing a moratorium on risky gene-splicing experiments in the 1970s. Furthermore, enterprise demand currently centers on practical, cost-effective models that can integrate into existing workflows, meaning AI companies could maintain immense commercial value by functioning as dependable technology providers rather than relentlessly racing toward superintelligence.
Amodei and other Silicon Valley leaders exhibit a strange contradiction: while acknowledging that researchers understand only a fraction of their models' inner workings and expressing concern over deceptive agents escaping containment, they still cling to the conviction that superintelligence is inevitable and worth the immense hazards. The author argues that inevitability is a flawed justification for pursuing research that could endanger civilization, concluding that Amodei's proposed slowdown should instead be a complete and decisive stop.