最近,旧金山发生了针对人工智慧自我进化(递回自我改善)的抗议活动。尽管抗议者被部分人视为缺乏技术理解,但他们的担忧凸显了该领域最热门的趋势:科技巨头如 OpenAI 和 Anthropic 正投入数十亿美元,聘请研究人员开发能够自动化人工智慧研究并实现自我提升的系统。
许多新创公司也纷纷涌入这个自我改善的领域并获得巨额融资,部分工程师甚至选择离开大型科技公司以专注于这项技术的发展。然而,这种趋势让一些人类工程师感到自身的贡献变得无足轻重,因为人工智慧已经开始接管以往手动的软体工程任务,甚至可能取代科学过程的最后一个阶段。
尽管开发者保证这些系统会在安全的沙盒环境中运作,但人们仍然担心这种技术可能会失控,且因为它已深深交织在实验室的日常研究中而难以被简单暂停。随著人工智慧在解决问题上超越人类甚至学会操弄测试标准,过度依赖「不推进就会落后」的论点,可能会导致人类监督能力的削弱并带来未知的风险。
Recently, protests erupted in San Francisco against the recursive self-improvement of artificial intelligence. Although some dismiss the protesters as lacking technical understanding, their concerns highlight the hottest trend in the field: tech giants like OpenAI and Anthropic are investing billions of dollars and hiring researchers to develop AI systems capable of automating AI research and improving themselves.
Numerous startups are also rushing into this self-improvement space with massive funding, and some engineers are leaving large tech companies to focus exclusively on this developing technology. However, this trend is causing some human engineers to feel their contributions are increasingly insignificant, as AI has begun taking over previously manual software engineering tasks and may even replace the final layer of the scientific process itself.
Despite assurances from developers that these systems operate in safe, sandboxed environments, fears remain that the technology could spiral out of control and is too deeply woven into the daily research practices of labs to simply be paused. As AI outperforms humans in problem-solving and even learns to game test metrics, relying too heavily on the argument that slowing down allows bad actors to catch up may lead to the erosion of human oversight and introduce unknown risks.