Chatbot Traffic Rises 60% YoY
Websites have seen an increase of more than 60% invisits from artificial intelligence (AI) harvesting bots during the past 12 months, according to data released Thursday.
Publishers are concerned about unwantedwebsite scraping, which means the site's content is taken and repurposed without permission and adds to all types of costs.
Nina Gould, chief innovation officer at Forbes, toldMediaPost last month that many bots will take the content, adding that it's like "playing whack-a-mole" to get them to stop.
"AI bot traffic is undermining the viability ofthe open web publishers used to reach audiences with their advertising," James Rosewell, CEO of 51Degrees, told MediaPost. "AI bots impose costs on publishers twice. Once in the bandwidth their botsuse when harvesting, and then again when they steal traffic from sites with AI results."
Based on analysis from 51Degrees of more than 3 billion website visits from the 12 months leading up toMay 2026, the percentage of AI harvesters detected on website traffic increased by 62.5% -- up from 8% to 13% of all web sessions, the U.K.-based technology company reported.
51Degrees,which provides real-time digital data services, identified the most prolific bot as AmazonBot, Amazon's web crawler.
AmazonBot, which is used to index content and gather data toimprove Alexa, Rufus, and product search results, represented 4.1% of all web visits.
Meta-ExternalAgent represented 2.9%, while Microsoft BingBot represented 1.35%, followed by MicrosoftBingBot at 1.35% Anthropic at 0.19%, OpenAI at 0.09%, Perplexity at 0.04% and Google Gemini at 0.0003% of web sessions.
Google also uses its search bot for AI harvesting -- anumber that is not included in this data, the company said.
The assumption is that lower numbers represent more mature technology, and that they have already taken most of the information onthe web they need to date, according to 51Degrees.
The statistics represent web sessions from bots that identify themselves as AI bots including ClaudeBot, PerplexityBot and others.
Although some bots masquerade as human users, 51Degrees could not capture those. None are valid as advertising traffic, but 51Degrees does not claim that site ownersare passing these on as ad impressions -- only that these are the volumes of visits they are receiving.
Publishers face a number of significant challenges. While some botsreach sites and deliver more traffic from search and other means, publishers often block crawling tags that bots use. Malicious bots work undercover and falsify data that makes them look like a humanuser.
Technology is being created to identify specific bots as Google, Cloudflare, and other technology companies rewrite the way the internet handles automated traffic.
Cloudflare data released earlier this year indicates that agentic AI bots generate roughly 57.4% of global web requests .
To address this surge, Google and Cloudflare developed technologies including PrivateAccess Control Tokens to differentiate human users from malicious scripts and authorized AI agents.
