The rise of the advanced persistent bots

Overall bad bot activity is decreasing, but the number of advanced persistent bots is up according to a new report.

The 2016 Bad Bot Landscape Report from Distil Networks reveals that 88 percent of all bad bot traffic has one or more characteristics of an advanced persistent bot, one that's able to mimic human activity and evade detection.

"When we dug into the bot activity in 2015, we identified an influx of Advanced Persistent Bots (APBs)", says Rami Essaid, co-founder and CEO of Distil Networks. "ABPs can mimic human behavior, load JavaScript and external assets, tamper with cookies, perform browser automation, and spoof IP addresses and user agents. The persistency aspect is that they evade detection with tactics like dynamic IP rotation from huge pools of IP addresses, use Tor networks and peer to peer proxies to obfuscate their origins, and distribute attacks over hundreds of thousands of IP addresses. A whopping 88 percent of 2015 bad bot traffic were APBs. This shows that bot architects have already taken note of traditional bot detection techniques and are finding new sophisticated ways to invade websites and APIs, in an effort to take advantage of critical assets and impact a business's bottom line".

Among the main findings of the report are that 46 percent of all web traffic originates from bots, with over 18 percent coming from bad bots. Medium-sized websites (those with a 10,001 to 50,000 Alexa ranking) are at greater risk, as bad bot traffic made up 26 percent of all web traffic for this group.

Chrome has edged out Firefox as the browser of choice for bad bot creators with over 26 percent of all user agents now using the Google browser. In addition 53 percent of bad bots are now able to load external resources like JavaScript meaning these bots can up falsely attributed as humans in Google analytics and other tools.

The report finds that 39 percent of bad bots are able to mimic human behavior, so tools such as WAFs, web log analysis, or firewalls, which perform less detailed analysis of clients and their behavior, will likely result in large amounts of false negatives.

It also finds that 36 percent of bad bots disguise themselves using two or more user agents, and the worst APBs change their identities over 100 times. Multiple IP addresses are used by 73 percent of bad bots to rotate or distribute their attacks, and of those, a surprising 20 percent used more than 100 IP addresses.

While six out of the top 20 ISPs with the highest percentage of bad bot traffic originated from China, Amazon has appeared in the top five bad bot originators three years in a row. It's still the US that's the largest originator of bots, with over 39 percent of bot traffic, while India and Israel moved up to the two and three spots.

You can find more information by downloading the full report from the Distil Networks website.

Photo Credit: Gunnar Assmy/Shutterstock