Filtration by User-Agents in the Anti-Crawler Option


We've extended the feature for working with user-agents for the Anti-Spam plugin.  This feature is available for the WordPress plugin only. To enable user-agent filtration please open your Anti-Spam plugin settings:

WordPress Dashboard -> Settings -> Anti-Spam by CleanTalk 

And enable SpamFireWall and Anti-Crawler option:


SpamFireWall option


Then open advanced settings (WordPress Dashboard -> Settings -> Anti-Spam by CleanTalk -> Advanced settings) and enable "Block bots by User-Agents":


Filter UA option


You can see User-Agent in your SpamFireWall logs when Anti-Crawler blocks the request. With "To Personal Black & White Lists" button you can add a User-Agent to your Personal Lists.

Please note that you can add user-agents existing in our list only. Google bot and Bing bot whitelisted by default, you can't add them to your personal lists.


SpamFireWall logs


You can manage your User-Agent Personal Lists here. Press "Add new filter" and set filter type as User-Agent:


User-agent filter


Then select agents from the drop-down list and Blacklist them:


User-agents list


Here is the list of all available User-Agents:

  • AhrefsBot
  • Akamai crawler
  • Alexabot
  • Applebot
  • archive.org_bot
  • AspiegelBot - HuaweiWebCatBot
  • (Barkrowler)
  • Baidu
  • Better Uptime Bot
  • Bing
  • BrightLocalBot
  • CleanTalk Uptime bot
  • CloudFlare crawler
  • DuckDuckGo
  • FacebookBot
  • FeedBurner
  • Feedly Fetcher
  • FreshpingBot
  • Google
  • Grapeshot
  • GTmetrix
  • GumGum-Bot
  • HuaweiWebCatBot
  • Hypefactors
  • Lighthouse
  • MailChimp
  • Majestic MJ12bot
  • MirrorWeb
  • Petalbot
  • bot
  • Pinterest bot
  • Printful WooCommerce Integration
  • proximic
  • Qwant Web crawler
  • Rambler bot
  • RankMathApi
  • RapidLoad (
  • Reddit
  • Rogerbot -
  • Screaming Frog SEO Spider
  • Semrush
  • SEOkicks-Robot
  • serpstatbot
  • Seznam
  • ShipStation
  • ShortPixel Image Optimizer
  • Site24x7
  • Sitechecker
  • sogou spider
  • StatusCake
  • Sucuri Uptime Monitor
  • Wikipedia crawler
  • WooCommerce API
  • wordoftravel: Find Travel Blogs and Explore Destinations
  • WP Engine Smart Plugin Manager
  • WP Rocket
  • WPCompress
  • Yandex

These User-Agents are system defined and can't be added to your Personal Lists:

  • Baidu
  • Bing
  • CleanTalk
  • Google
  • Yandex



If you haven't found the answer to your question, please, contact our support team:



Was this information helpful?



Perhaps it would also be interesting