Targeting Bot traffic
This article will cover the following:
- Using a user-agent based rule to target bot traffic
- Using an ISP provided-based rule to target bot traffic
There are multiple ways to target bot traffic. Bots are generally not confined to a single IP so that you can block that IP. Instead, you'll just find that they continue to come in under different IP's. To get ahead of this there are 2 approaches to handling bot traffic - with a 1) user-agent based rule and a 2) ISP or provider-based rule.
Using a user-agent based rule
This method is advised when you can see a consistent user agent that's unique to the bot and can be identified with a unique phrase or word. This will not work if the user agents look like generic browser
- Change targeting method for your contract to rules targeting by clicking on your Contract > click Change Targeting Method > select Rules > then click Apply. This should give you the target-looking icon.
- In the rules window that pops up, select New Rule
- Then assign a name (try to be as specific as possible so it's not confusing for you later)
- Check Active
- Shared – up to you (if you think you'll want to use it later then go ahead and select Shared. This can be updated at any time)
- Choose your redirect offer or global redirect or 404
- Then in the bottom window, select Add Rule Target
- Select User Agent
- In the user agent window that pops up, enter the search phrases you want to identify and include the search phrases. For example if I wanted to identify any user agent that either AcoonBot or Baiduspider, I would add "AcoonBot|Baiduspider" (pipe-delimited)
- Click Save
- Click Create
Using an ISP provider-based rule
You'll know when you need to use this rule because the user agents look just like generic browser user agents. What will be consistent across all those clicks is the Provider.
- Rules 1-7 stay the same as above.
- When it comes to selecting your rule target, select ISP/Carrier
- In the ISP/Carrier window that pops up, click Add Group
- In the Manage Group window, search for the relevant term and add the appropriate providers, then click Save