← 返回 Avalaches

近期人工智慧领域的领军人物相继呼吁放缓技术研发步伐。Anthropic 执行长 Dario Amodei、OpenAI 的 Sam Altman 以及 Elon Musk 等人皆表态支持限制 AI 推进速度。这项倡议源于对「递归自我改进」以及自主智能体群体可能发动未授权网路攻击等潜在失控风险的深切担忧。尽管外界质疑这可能是科技巨头试图建立垄断卡特尔或防御外部监管的手段,但作者认为,如同核能与制药产业,具备巨大潜在破坏力的领域接受适当监管具备充分的理性基础。

面对技术风险与企业间及跨国间的激烈竞争,Amodei 提出了派遣嵌入式评估员、促进企业间民主协调以及开展全球政府合作等三步骤方案。与此同时,美国前财政部长 Hank Paulson 和 Robert Rubin 亦倡议美中两国应致力于签署《人工智慧合作条约》(ACT),并成立类似金融市场工作小组的总统级 AI 工作小组,以在避开反垄断疑虑的同时凝聚产官共识。然而,川普等政治人物对此类风险展现出的冷漠,加上对国家安全与国际竞争的执著,使跨国协同监管的推动充满严峻阻碍。

作者预测,由于商业竞争动能难以遏制以及国家安全考量带来的政策犹豫,短期内各国政府将难以建立全面且有效的监管机制,人类可能仍将盲目冒进并心存侥幸。针对未来的演变路径,作者指出三种情境:一是 AI 的灾难性风险被过度夸大,冲击仍在可控范围;二是先爆发一场可承受的灾难,迫使全球仿效过往药品监管历史建立严厉的究责与审查体系;三是灾难预言不幸成真,届时将因未及时防范而蒙受难以挽回的毁灭性后果。





Leading figures in the artificial intelligence sector, including Anthropic's Dario Amodei, OpenAI's Sam Altman, and Elon Musk, have increasingly called for a deceleration in the pace of AI model capabilities. These appeals are driven by acute concerns over recursive self-improvement and uncontrolled autonomous agent swarms, which threaten to surpass human comprehension and oversight. While skeptics suspect ulterior motives such as entrenching market dominance or preempting external scrutiny, the author argues that industries possessing vast potential for harm, much like banking, pharmaceuticals, and nuclear energy, rationally warrant stringent regulation.

To confront the twin obstacles of fierce corporate and geopolitical rivalry, Amodei proposed embedded safety evaluators, intercompany democratic coordination on standards, and global verification agreements between democratic and authoritarian states. Concurrently, former US Treasury Secretaries Hank Paulson and Robert Rubin urged the establishment of an AI Cooperation Treaty between the US and China alongside a dedicated presidential working group to foster collaborative solutions without antitrust hindrances. Nonetheless, widespread political indifference, exemplified by Donald Trump's prioritization of national competition over safety, presents formidable friction to achieving such coordinated oversight.

The author anticipates that competitive commercial pressures and the prevailing national security narrative will likely paralyze meaningful regulatory action, leaving society to advance haphazardly while merely hoping for favorable outcomes. Looking forward, the author outlines three potential trajectories: the apocalyptic risks may ultimately prove overblown and manageable; a survivable catastrophe could occur that compels immediate, Thalidomide-style regulatory overhauls under immense public pressure; or the catastrophic warnings will materialize when it is already far too late, highlighting the perilous folly of failing to implement precautions in advance.
2026-09-17 (Thursday) · 8335efb0bb71bdab2263fa13ba67da3541baf925

Attachments