Cloudflare SSL blocking twitterbot

Hello, my pages won’t validate on Twitter Card Validator. I started a threat on their support board and this is what their support staffer said:

I see a 403 Forbidden error when I try to fetch them as Twitterbot, but if I drop the explicit user-agent (ie. I pretend to be a browser like Chrome) then the requests work. The 403 is coming directly from CloudFlare so I suspect there might be something up with the configuration there.

I have a robots.txt allowing twitterbot access to everything. Any idea how CloudFlare is blocking it? Thanks for any help.

The url is frankmanuel (dot) com

That wouldn’t be SSL, but maybe something in the Firewall. Check the Firewall Events Log to see why it might be blocked.

Thanks, I just checked the firewall and there’s a few Hotlink protection actions but the user agents are all browsers and the image is my favicon for some reason. Nothing about twitterbot.

Does anyone know what I can do about this, or know how I can contact someone that works at Cloudflare? Thanks!

I went into Firewall page rules and set a rule to Allow User Agent Twitterbot. It hasn’t helped. When I use Google Chrome’s developer tools to check as user agent twitterbot, the page has 403 Forbidden. Please, can someone tell me how to give Twitterbot permissions aside from what I’ve already done, which is giving a page rule to allow it and using robots.txt to grant it access to everything? Or can someone please tell me how I can get a hold of someone from Cloudflare? Thanks.

Ok, solved. My host says they’re the ones blocking Twitterbot, not Cloudflare, due to abuse by the bot. I’ll have to switch hosts. Not happening on Cloudflare’s end. Thanks.

2 Likes

This topic was automatically closed after 30 days. New replies are no longer allowed.