How to allow ads bidding bots

Website, Application, Performance Security
1

Hi, I have a rule that challenges all direct traffic so some good unknown bots are blocked.

How can I allow ads bidding bots such as the two below? Is there anything else I should allow?

CriteoBot/0.1 (+https://www.criteo.com/criteo-crawler/)

Mozilla/5.0 (compatible; TTD-Content; +https://www.thetradedesk.com/general/ttd-content)

2

Greetings,

Thank you for asking.

Using a Firewall Rule like if User-agent contains criteo or User-agent conzains ttd-content, method “allow” and make sure it’s the 1st rule from the above on the Firewall Rules list.

In picture as an example:

example
example1625Ă—1273 122 KB

Firewall Rule expression:

  • (http.user_agent contains "criteo") or (http.user_agent contains "ttd-content")

Otherwise, if you experience it’s sill being blocked or challenged and does show up at your Security → Overview as a Firewall Events, you could allow Criteo ASN by adding AS44788 (ASN-CRITEO-EUROPE) to the Security → WAF → Tools → IP Access Rules with the action “allow” for your Website.

Also, it’s not a “good verified bot” yet, so I think there could be some catches:

example_ipar
example_ipar1606Ă—743 25.2 KB

Useful articles:

2 Likes
3

Thank you very much. Do I have to find all ads bidding bots and allow them one by one? I searched online but couldn’t find a list of them.

Is it safe to allow path “/ads.txt”?

4

This topic was automatically closed 15 days after the last reply. New replies are no longer allowed.