Known Bots rule is blocking some google crawlers

Continuing the discussion from: https://community.cloudflare.com/t/known-bots-rule-lacks-many-google-crawler-ips/469572

The “Known Bots” field in the WAF wasn’t working properly for the user, leading to Google crawler getting blocked with a “403” error. The author had to manually add google IPs to a new WAF rule.

I asked them for more information and logs.

Ticket number: 2649240

2 Likes

There is an ongoing CUSTESC-25204 for this user.
Little of back and forth with eng.

Will update in here once we get more information with them.

3 Likes

In the meantime I already responded the customer in the ticket and ask him to put back the allow list rule for known bot.

1 Like

Customer has stated they have added the rule/s requested.

Pending further details from customer or Rommy once a reply is sent.

Bots team is still reviewing this under CUSTESC-25204.

Escalation CUSTESC-25204 is still being worked on

I’ve chased Engineering and Product in the CUSTESC-25204, and I’ve updated the customer in the ticket.
Strange issue… Took me a long time to digest the ticket and the Jira… :sweat_smile:

3 Likes

The CUSTESC-25204 is still being worked on. Awaiting on engineering team to clarify root cause and suggested workaround.

1 Like

The Engineering team has deemed this as a known issue: O2O has not been certified as working with Bot Management.
At this time this time the customer applied the suggested workaround (unproxying relevant DNS record).
It is being further tested and discussed through FPOS-161 and ERE-322, but I reckon no real ETA’s in sight yet.

1 Like

This topic was automatically closed 3 days after the last reply. New replies are no longer allowed.