Bots: Do You Know Why They're Important?

Many digital transactions are carried out not by humans, but by automated software tools (“bots”) masquerading as humans for a variety of reasons; some legitimate and some not-so.

A bot is a piece of software used to run automated tasks on the internet, normally deployed to remove the need for human intervention in everyday operations. Typically, they mimic human behaviour and are used to replace repetitive and time-consuming tasks as they can perform them at a much faster rate. However, they can also be used by fraudsters to fake human-looking activity for several criminal purposes.

As reported in Search Engine Watch in February 2019 “$7.2 billion was lost to click fraud between 2016 and 2018”. Cybersecurity company Cheq are quoted in The Drum in June 2019 as saying that “the direct cost [of click fraud] to advertisers to hit $26bn in 2020, $29bn in 2021, and $32bn in 2022” noting that “This burden lands disproportionately on the shoulders of small firms that don’t have as much money as established brands to avoid ad fraud”

Non-human digital activity leads to a global issue around online advertising fraud; for marketers, it continues to have a huge negative impact on planning, undertaking and reporting of accurate digital activity.

What are the different types of bot?

Bots generally fall into four distinct types:

Web Crawlers: The most common type of bot, also known as spider bots, these like Googlebot, scan the internet to find new content on webpages and add it to global indexes.

Chatbots: These are used to simulate human conversation by responding to key phrases with programmed responses which walk customers through a series of interactions. Used by many companies for the first line of customer support.

Social Bots: These operate on social media platforms like Twitter and are used to distribute information or operate news feeds. They can be both benign or malignant.

Malicious Bots: Malicious bots are normally trying to obtain data which can be used for fraudulent purposes, to steal money, divert assets, disseminate false information or carry out Denial Of Service attacks to hold businesses to ransom. They are also used to generate online profiles that look they are human, for the purposes of generating revenue from paid media (PPC, advert and promoted social posts).

As well as losses to individual companies, bad bots are becoming increasingly more common to fund crime, especially drugs and human trafficking, property fraud and child exploitation.

Why Do Criminals Run Bots?

One of the most famous rogue bots was Methbot, an advertising fraud scheme which was first tracked by cybersecurity firm White Ops in 2015.

Controlled by a single group based in Russia and operating out of data centres in the US and Netherlands, its bot farm was generating between $3 and $5 million in counterfeit inventory per day by targeting the premium video advertising ecosystem.

Working in partnership with Trustworthy Accountability Group (TAG) and the Interactive Advertising Bureau’s Tech Lab, White Ops released all their research into the bot allowing advertisers, agencies, social media platforms, and publishers access to the necessary data to stop it operating.

In 2017 two of the largest criminal Dark Web markets associated with Methbot, AlphaBay and Hansa, were taken down following a collaboration between the US’s Federal Bureau of Investigation, the US Drug Enforcement Agency (DEA) and the Dutch National Police, with the support of Europol.

The takedown removed the infrastructure of an underground criminal economy responsible for the trading of more than 350,000 illicit commodities including drugs, firearms and cybercrime malware.

The UK’s National Crime Agency released a report in 2018 - National Strategic Assessment of Serious and Organised Crime - which provided ‘a single picture of the threat to the UK from serious and organised crime’.

The report assessed threats in a variety of areas and grouped them together into three ‘pillars’ of response (Vulnerabilities, Prosperity, Commodities), with aspects which cut across multiple threats captured separately.

The NCA believes the threat from Serious and Organised Crime (SOC) is increasing in both volume and complexity and will continue to do so in the short to medium term.

What Steps Can I Take Against Bad Bots?

Recent research on “Internet advertising: Reliability, Dilemmas, and Possible Directions” by Huddersfield University, UK, in conjunction with Bin Faisal University, UAE and Digital Data Specialists, Beaconsoft Ltd, looks at the taxonomy of click fraud attacks, main click fraud perpetrators, contemporary countermeasures techniques and the viability of data mining and machine learning approach for detecting click fraud:

“Several methods could be employed for detecting ad bots. The majority of current ad botnet detection techniques typically depend on Deep Packet Inspection (DPI) by analysing the packet payload (contents)...However, this technique might not deliver an in-depth

analysis of all acquired traffic...mitigating ad bots is still an issue of on-going attention for academic research as well as the professional community.”

This makes the point that the detection of bot traffic is very complex and there are lots of players who are involved with trying to solve this; however, there are a number of steps you can take yourself to get ahead of the game, which will be expanded upon in the DMA North Council’s forthcoming white paper on Artificial Engagement.

Three steps you can take are:

1. Monitor your digital media campaigns for fraudulent traffic - which is to say engagement with promoted posts, shares and visits to you or your clients’ website by bots.

2. Assessing the locations and IP addresses of web visitors, unusual spikes in bandwidth, traffic from countries outside the normal visitor locations, and visitor behaviour on site.

3. Once you have identified suspicious activity, use Google Ads’ built-in IP blacklist functionality. Facebook ads are more complicated, as their own system prevents IP address blocking, so you will need to identify the facebook_id of the suspected bot first. Tools exist to support you with this, including Beacon.

It’s important to build an internal culture of not being prepared to accept fraudulent media spend in your campaigns: ensure that you continually evaluate your own data and data reported to you by agencies, if you have one, and ad networks. Hold them and yourself to account. Money wasted on bots is wasted twice: firstly because you’ve spent money on something that can never pay you back, and secondly, because it allows criminal organisations to carry out more nefarious activity that is bad for society.

About The Author

Stewart Boutcher is a member of the DMA North Council, Chair of the DMA Leeds Hub and Founding Director and CTO of Beaconsoft Ltd, a UK-based company specialising in digital campaign intelligence.

Beaconsoft was set up in 2017 to developed Beacon, a digital analytics platform which focuses on identifying false clicks and optimising digital spend based on true results. When it comes to social media and digital marketing, using Beacon allows you to syphon out a lot of the fake clicks and fraudulent bot traffic.