Accelerate your thought leadership by contributing to our blog. Join our community of experts now!
Enjoy reading this blog post written by our experts or partners.
If you want to see what Databox can do for you, click here.
Some reports say that over half of all Internet traffic is bots.
Contrary to popular belief, bot traffic isn’t always bad.
There are some good bots like the ones that crawl and index websites on search engines or power our voice assistants like Siri and Alexa.
But, there are also malicious bots that can cause major headaches for website owners like DDOS attacks.
The key is to get better at identifying and filtering out bot traffic so that you have reliable Google Analytics data on what humans – not bots – are actually doing on your website.
In this post, we’re taking a closer look at how to identify and filter out bot traffic in Google Analytics, including:
Bot traffic is any traffic that is automated and doesn’t come from humans.
This affects all websites on the Internet, from popular sites with millions of visitors each week to smaller websites with fewer than 10,000 monthly visitors, as we can also conclude from our survey.
Out of the marketers we surveyed, 25.7% say that they experience bot traffic on a daily basis.
So, what portion of your website traffic is actually bot traffic? Let’s find out how can you discover this in tools like Google Analytics.
Who has visited your website, human or bot? Fortunately, there are some telltale signs to identify bots in your Google Analytics reports.
Sure, there are dozens (and dozens?) more Google Analytics 4 metrics you could track. But, starting with these 10 commonly tracked metrics will give you a pretty high-level view of how your marketing is working…
If you want to track these in Google Analytics, you might find the visualizations limiting. It’s also a bit time-consuming to combine all the metrics you need in one view.
To better understand how your website performs in terms of traffic growth and conversions, we’ve made this plug-and-play dashboard that contains all the essential metrics for understanding how successful you are at optimizing different aspects of your website.
You can easily set it up in just a few clicks – no coding required.
To set up the dashboard, follow these 3 simple steps:
Step 1: Get the template
Step 2: Connect your Google Analytics account with Databox.
Step 3: Watch your dashboard populate in seconds.
If your website normally gets 1,000 visitors per day and one day you get 20,000 visitors without doing anything different from a sales or marketing perspective, this is most likely bot traffic.
“Probably the most obvious way is to review traffic patterns for unexpected spikes,” explains Donna Duncan of B-SeenOnTop. “If the traffic can be isolated and consists primarily of single page views of 1 second or less, I dig further. Often the city will be not set or the location of a server farm (like Boardman or Ashburn) or the keyword searched for or referring source will be a domain containing the word bot.
Ben Johnston of Sagefrog Marketing Group says, “The best way to find bot traffic is to look for spikes on a certain day, or series of days and look at the behavior metrics. A majority of the time, bots will have an average session duration that is less than a second, and will only have a 1 page per session. We’ve had success filtering out this traffic by looking at keywords in the organic channel (where we’ve seen the majority of recent bot traffic), and finding the bot-related keywords. The keywords we have found are something like “bottrafficxyz.com” which can be easily filtered out using an advanced filter.”
Andy Crestodina of Orbit Media Studios adds, “When you see a spike that can’t be otherwise explained, look for evidence that it’s a bot. In this case, it was traffic to a non-page (so-called ghost traffic injected directly into GA without actual triggering of javascript on the site)
When you plot the row, you can see the spike clearly.
Next just exclude them with a quick conditional segment:
Filter Sessions > Exclude > Page > Contains > (garbage URL)
That’s it! Bots removed. Now everything in Analytics will look normal again as you navigation around the reports. Your bounce rates, conversion rates are more accurate again.”
Want to learn more? Andy Crestodina of Orbit Media Studios made a video that shows how to create and use one of the most helpful Google Analytics features for identifying bot traffic and analyzing your data – GA segments.
Another tactic for identifying bots is to dive into your Campaigns report in Google Analytics.
Zack Duncan of Root and Branch Group explains, “Our bot traffic often shows up in our Campaign reports in Universal Analytics (Acquisition -> Campaigns -> All Campaigns). For example, I see a campaign named ‘cvgg_dybff’ that I definitely did not set up, along with the UTM tagged campaigns that I did set up like ‘buttonclickvideo’ and ‘gmb.’
Once something shows up there that is clearly a bot, I can create a custom filter to exclude any other traffic with the same filter pattern. The filter field is ‘Campaign Name.’
I’ll also investigate that bot traffic further by adding secondary dimensions to see if there are any other filters that I may want to apply. For example, the Source of the spammy campaign may show as bottraffic4free\.club, in which case I’d add another filter with filter field = ‘Campaign Source’ to exclude that source in the future.”
Another tactic is to monitor your referral traffic sources closely.
“One way bot traffic often comes in is through referral traffic,” says Elizabeth Weatherby of Ocean 44. “If you see some funky bots in your referral traffic channel, and you know these bots are not actual referral traffic, you can filter them out through the referral exclusion list. This is a simple and easy way to help rid your site from bots coming through referral traffic!”
Tom Zsomborgi of Kinsta says, “I recommend going through the list of referrals (Acquisition/All traffic/Referrals ) at least 1-2 times each month. If you see some new websites there with a spammy looking name and high visits next to them you can manually check these website and add to your list of banned sites. After that, you can create a filter in GA and add these spam sites.xa”
Bernadett Kovacs-Dioszegi of Creatopy adds, “It’s important to check if your site receives referral spam, which is created by spambots. These spam bots are created to visit websites to fake a pageview. However, there are some good bots, like ones that are monitoring your website’s health.
When it comes to filtering out referral spam you should make a difference between Universal Analytics and Google Analytics 4. In case of Universal Analytics, before you start to test new filters, you should always create a new view. It’s not recommended to make these kinds of changes directly in your default raw data, because filter changes are harmful and irreversible. They aren’t retroactive either.
Instead of making a referral spam blacklist, which can be time-consuming and even if you exclude the referral, you will continue to receive invalid traffic, that’s the reason why some people noticed an unexpected increase in direct traffic, it’s better to create a filter including real hostnames. This will be your domain of the majority of cases. However, it may also result from translation services, or anywhere where GA tracking code has been inserted. The list of these hostnames you are able to check in Universal Analytics, by clicking on Audience in the left-hand panel, select Network from Technology tab and at the top of the report, click on Hostname. After this you can build a regular expression similar to this one: yourmaindomain\.com|anotheruseddomain\.com|payingservice\.com|translatetool\.com
Once you have created the regular expression, it is time to build your custom filter in the UA admin. Make sure you select to include the hostname filter field.
Theoretically in Google Analytics 4 properties, traffic from known bots and spiders is automatically excluded, but practically there are only two filter types available: Developer Traffic and Internal Traffic. Referral exclusions are not implemented yet.”
Another telltale sign of bot traffic is an influx in visitors that stay for one second or less.
“One way that we filter out bot traffic is to drop filter out any of sessions that lasted less than 1 second,” says William Chin of ProlightingRental.com. “Usually, bot requests don’t have the capability to trigger actual time on page (so, therefore, will return 0:00 seconds – but count as a page view). You can also consider bringing up that threshold to 1 or 2 seconds as a real user would likely NEVER engage with a web page that little.”
In addition, here are some tips for how to clean up your Google Analytics data and filter out spambots.
Before you start excluding bot traffic, it is a good idea to create a new view in Google Analytics.
“I’d first recommend setting up an Unfiltered View in Google Analytics to avoid losing traffic data by inadvertently blocking legitimate bots,” says Sam Gooch of Kinsta.
Next, create a new View that can be used to exclude spam. Just head over to the Filters section in your new View from the Admin dashboard. There are many filters that can be created to block different types of spam, but here’s one that can catch the most common offenders when it comes to crawler spam. This type of spam comes from bots that browse websites and log information by performing automated tasks on websites. These bots ignore rules set up in your robots.txt file, giving them free rein over your site. To set this up, create 2 filters, choosing “Custom” for the Filter Type, and then select “exclude.” Set field equal to “campaign source” then add the expression below into the box, repeating the process for the second filter.
This will exclude traffic from these known bots, which can help to prevent your data from being skewed.”
There is actually an admin setting that allows you to filter a significant chunk of bot traffic.
“In the admin section, if you navigate to the view column, click view settings, and scroll to the bottom of the page you’ll see a tickbox that says ‘Exclude all hits from known bots and spiders,’ says Alistair Dodds of Ever Increasing Circles. “Make sure to select this option and then click the save button. Google Analytics will now filter all bot and spider traffic so you see the true numbers of human visitors.”
Maggie Simmons of Max Effect Marketing says, “Because of how this is implemented, you won’t know if bots are entering your website, and you won’t be able to filter specific bots. If you want to exclude them, though, you’ll note a significant shift in your traffic patterns, which may help you better understand your website visitors.”
Alex Birkett of Blissfully explains, “Bot traffic changes constantly, which makes filtering it out frustrating. So, you can start out by going to View Settings and then Bot Filtering”in your Google Analytics account and then hitting Exclude all hits from known bots and spiders. But that only reduces a percentage of bot traffic. If you’re serious about cleaning things up, set a time window to audit your analytics data (I like quarterly) and diagnose and identify bot traffic. Then add those to your exclusion parameters in your Google Analytics filters once identified.
For example, you can create filters to exclude bot traffic from specific landing pages
Chad Hadley of Power Digital Marketing explains, “I used a Segment to filter out bot traffic from my clients’ GA. I set it up using CONDITIONS as such:
• Filter: Sessions > Exclude
• Landing Page > Starts with > [BOT]
If there is more than one bot discovered (either on the same client or another), I would then add subsequent Filters excluding them to create a comprehensive segment.”
You can also set up filters and segments by source/medium.
“It’s easy to create a custom segment in Google Analytics that filters out bot traffic,” says Bruce Hogan of SoftwarePundit. “This can be done using the source field. For example, excluding all traffic where source = trafficbot.”
Veronica Miller of VPNoverview adds, “To detect if your website has visits from bots, access your Google Analytics dashboard; on the left side menu, click Acquisition > All traffic > Source / Media.
To identify which accesses are from these bots, look for referral traffic with different or unknown names; seeing these visits, you will know which are the ones that visit your virtual store frequently. Now that you have identified any nonexistent visits, let’s create a filter so that it is not counted in the reports.
Now, in the left side menu, in the view, not in the main menu, click on Filters. Now, click on “”+ Add a new filter at the top of the form.”
A form will open to be filled, but first, click on the Custom option just below the filter title. Now, fill in the form as follows:
After filling in these fields, do not select anything else and click on Save.”
You can also create filters by specific hostnames.
Shaun Elley of Blue Ox Digital LLC explains, “Find the hostname of the bot traffic by adding a second dimension for hostname on the source medium report to make sure it’s bot traffic and then create an include filter for your own hostname.”
“Most Google Analytics spam never actually visit your site,” says David Gossage of LOOKFANTASTIC. “It just it triggers random tracking codes in the hope that some of them will get a hit. This is known as “”ghost spam””. The following trick will block all bots that do not crawl your actual site and will remove a massive chunk of spam traffic in one go without having to manually block many different domains.
In the Admin section, create a filter for the view and use the following settings:
Include only > Hostname > That contain > yourdomain.com
That’s it.”
In sum, bot traffic doesn’t have to ruin your Google Analytics reports. These tips can help you quickly identify and filter any bots from your data.
Hey, we’re Databox.Our mission is to help businesses save time and grow faster. Click here to see our platform in action.
Are you maximizing your business potential? Stop guessing and start comparing with companies like yours.
At Databox, we’re obsessed with helping companies more easily monitor, analyze, and report their results. Whether it’s the resources we put into building and maintaining integrations with 100+ popular marketing tools, enabling customizability of charts, dashboards, and reports, or building functionality to make analysis, benchmarking, and forecasting easier, we’re constantly trying to find ways to help our customers save time and deliver better results.
Jessica Malnik is a content strategist and copywriter for SaaS and productized service businesses. Her writing has appeared on The Next Web, Social Media Examiner, SEMRush, CMX, Help Scout, Convince & Convert, and many other sites.
Get practical strategies that drive consistent growth