How do I always allow a known bot client to index my site

Website, Application, Performance Security
1

My site’s internal search is powered by Swiftype (an Elastic Search product) but when I recently enabled my Bot settings to “Manage a Challenge” of “Likely Automated” traffic, it blocked the index. After some troubleshooting I discovered that it was returning a 403 error.

Is there a way to allow a certain user agent so I can keep the “Managed Challenge” setting on “Likely Automated” traffic, while allowing this User-Agent. The user agent is known. It is:

Mozilla/5.0 (compatible; Swiftbot/1.0; UID/5c40f8bc8db2314527c1b35f; +http://swiftype.com/swiftbot)

1 Like
4

For Free plans, toggle Bot Fight Mode option to Off under Security > Bots

For Pro, Business and Enterprise (no Bot Management add-on) plans: use the Skip action in Custom Rules to specify where Super Bot Fight Mode should not run. Skip rules are only available to customers who have migrated from Custom Rules to Firewall Rules.

2 Likes
5

This topic was automatically closed 15 days after the last reply. New replies are no longer allowed.