Whitelisting Search Atlas Bot

Table of Contents
a person working on a laptop with graphs and charts on the screen in a bright, modern office environment.

How to allow Search Atlas’s User Agent in Cloudflare

If you want to run a site audit and monitor your site health using Search Atlas, sometimes our site crawlers get blocked by CDNs.

Content Delivery Networks (CDNs) are a group of servers that help deliver website content faster. If your website relies on a CDN to increase the performance of your website, then it’s possible the CDN’s firewall will block our crawler from accessing your site.

Resolving this issue requires turning off bot-blocking features, as Search Atlas’s crawler doesn’t use static IP addresses. This post will explain how to do so on Cloudflare, as it is the most popular CDN provider for webmasters.

If you are using another CDN, the process is similar, and can still be done by following the steps outlined below.

How to allow Search Atlas’s User Agent in Cloudflare

Follow these steps:

  1. Login to your Cloudflare account
  2. Click Security and select the Bots submenu
  3. Toggle off either Bot Fight Mode or Super Bot Fight Mode

After performing the above steps, Search Atlas will have access to your website for monitoring.

For help and a better understanding of Cloudflare’s IP Access Rules, see Cloudflare’s support documentation.

How to allow Search Atlas’s User Agent in WordFence (WordPress)

You can allow the Search Atlas site crawler in WordFence by following these steps:

  1. On the WordPress menu, go to WordFenceFirewallFirewall Options
  2. Set the “Web Application Firewall Status” to “Disabled”
  3. Click the “Save Changes” button.

How to allow Search Atlas’s User Agent in Stop Bad Bots (WordPress)

Follow these steps to allow the Search Atlas crawler:

  1. On the WordPress menu, go to Block Bad BotsSettings
  2. Go to the Settings tab
  3. Disable the “Block All Bots” option, or add our user agent Search Atlas Bot (https://www.searchatlas.com/)to the allowed list.

How to allow Search Atlas’s User Agent in Blackhole for Bad Bots (WordPress)

To allow Search Atlas’s crawler in the Blackhole plugin:

  1. On the WordPress menu, go to BlackholeSettings
  2. In the Whitelist User Agents section, add the Search Atlas Bot (https://www.searchatlas.com/)user agent.
  3. Save changes.

How to allow Search Atlas’s User Agent in Shield Security (WordPress)

If you use Shield Security, follow these steps to allow the crawler:

  1. On the WordPress menu, go to Shield SecurityIP Rules
  2. Use the gear icon for additional options and select Create New User Agent Rule
  3. Add the Search Atlas Bot (https://www.searchatlas.com) user agent to the bypass/whitelist list.
  4. Click Create New Rule.

Need Help?

If you are still unable to successfully run a site audit after attempting the above methods, please contact our support team between 9am-6pm (PST), M-F, via the chat feature in your dashboard. Or, book a software demo with someone from our team.

Did you find this article helpful and accurate?

Thanks for your feedback!

Like this article? Share it!

Picture of Manick Bhan
CTO/Founder
Manick Bhan is the Founder/CTO at LinkGraph. He is also the creator of the enterprise SEO software suite, Search Atlas. He has over 10 years of experience helping Fortune 500 companies and enterprise brands scale their organic presence and earn revenue through data-driven SEO.

The #1 SEO platform for site owners, digital marketers, and SEO professionals.

Did you find this article helpful and accurate?

Thanks for your feedback!