Google 长期以来透过搜寻引擎的网路爬虫来建立索引，借此为网站带来人类读者的流量，但如今这项策略已经转变为同时收集资料来训练其人工智慧模型。这种将搜寻与人工智慧资料收集捆绑在一起的做法，不仅让 Google 获得了不公平的竞争优势，也让网站发布者陷入两难，因为他们无法在不流失搜寻流量的情况下单独阻止人工智慧的抓取。

这种由机器自动产生的网路流量逐渐超越人类活动，引发了对于原创内容经济诱因消失的担忧。随著人工智慧越来越多地读取、摘要并改写网页内容，而不再将读者导回原始网站，未来的网际网路可能会面临缺乏原创性与实用性的危机。

面对来自资安公司 Cloudflare 与英国竞争及市场管理局的强烈反弹，Google 目前正在测试一项新设定，允许网站选择退出人工智慧的资料抓取，且不会影响其搜寻排名。然而，专家认为更彻底的解决方案应该是将搜寻索引与人工智慧抓取的爬虫程式完全分离，让网站拥有更自主的控制权。

Google has long used web crawlers to index the internet and drive human traffic to websites, but this strategy has recently shifted to simultaneously collecting data for training its artificial intelligence models. This bundling of search indexing and AI data scraping not only gives Google an unfair competitive advantage but also creates a dilemma for publishers, who cannot block AI scraping without simultaneously losing their essential search traffic.

The increasing dominance of automated machine traffic over human activity raises concerns about the disappearance of economic incentives for original content creation. As AI increasingly reads, summarizes, and rewrites web content without directing readers back to the source, the future internet faces the risk of becoming devoid of originality and usefulness.

Facing strong pushback from web security firm Cloudflare and the UK's Competition and Markets Authority, Google is currently testing a new setting that allows websites to opt out of AI scraping without hurting their search rankings. However, experts argue that a more thorough solution would be to completely separate the crawlers used for search indexing and AI training, giving websites more autonomous control over their data.