Filtration by User-Agents in the Anti-Crawler Option

 

The Anti-Spam plugin for WordPress allows you to filter bot access to your site by User-Agent. To enable this feature, open your Anti-Spam plugin settings:

WordPress Dashboard → Settings → Anti-Spam by CleanTalk → Advanced settings

And enable the SpamFireWall and Anti-Crawler options:

SpamFireWall & Anti-Crawler options

There is a comprehensive list of trusted User-Agents in our service, whose requests will not be blocked by Anti-Crawler. If you want to block access to the site for any of them, you can add them to your personal blocklists.

To do this, click "Add New Filter" on the SpamFireWall personal lists page and select the User-Agent type. Then select an user-agent from the drop-down list and block it:

List of User-agents in personal lists

List of all available User-Agents (spoiler)

 

  • admantx.com
  • AhrefsBot
  • Akamai crawler
  • Alexabot
  • Amazonbot
  • AndiBot
  • Applebot
  • archive.org_bot
  • AspiegelBot - HuaweiWebCatBot
  • Babbar.tech (Barkrowler)
  • Baidu
  • Better Uptime Bot
  • Bing
  • Bravebot
  • BrightLocalBot
  • CleanTalk Uptime bot
  • CloudFlare crawler
  • DuckDuckGo
  • FacebookBot
  • FeedBurner
  • Feedly Fetcher
  • FreshpingBot
  • Google
  • Grapeshot
  • GTmetrix
  • GumGum-Bot
  • HuaweiWebCatBot
  • Hypefactors
  • KomoBot
  • Lighthouse
  • Mail.ru
  • MailChimp
  • Majestic MJ12bot
  • Meetedgar.com
  • MirrorWeb
  • Petalbot
  • PhindBot
  • Pingdom.com bot
  • Pinterest bot
  • Printful WooCommerce Integration
  • proximic
  • Qwant Web crawler
  • Rambler bot
  • RankMathApi
  • RapidLoad (rapidload.io)
  • Reddit
  • Rogerbot - MOZ.com
  • Screaming Frog SEO Spider
  • Semrush
  • SEOkicks-Robot
  • serpstatbot
  • Seznam
  • ShipStation
  • ShortPixel Image Optimizer
  • Site24x7
  • Sitechecker
  • sogou spider
  • StatusCake
  • Sucuri Uptime Monitor
  • UptimeRobot.com
  • WaldoBot
  • Wikipedia crawler
  • WooCommerce API
  • wordoftravel: Find Travel Blogs and Explore Destinations
  • WP Engine Smart Plugin Manager
  • WP Rocket
  • WPCompress
  • Yandex
  • YouBot

 

Important! It is not possible to block or allow User-Agents that are not on this list.

 

These User-Agents are set by the system and cannot be blocked by personal lists (spoiler)

 

  • Baidu
  • Bing
  • CleanTalk
  • Google
  • Yandex

 

 

If you want to add any User-Agent to the list of allowed ones, or if you haven't found the answer to your question, please, contact our support team:

https://cleantalk.org/my/support/open

 

 

Was this information helpful?

It would also be interesting

Copied to clipboard