How can i solve with CF rate limiting the Facebook scrapper attacks?

[fritex]

@hypercentergr

Facebook ips attack my server

@hypercentergr

scrap

I am sorry to hear this.
Unfortunately it’s a know issue that they’re very aggressive.

@hypercentergr

How could i rate limit the facebook attacks?

Create a Rate Limiting rule which checks URI Path wildcard, for 10 requests per 10 seconds with action block for 10 seconds (at least) - if you’re on a Free plan at least.

Create a rate limiting rule in the dashboard | Cloudflare Web Application...

If you’re on a paid plan, check what’s available to you:

Rate limiting rules | Cloudflare Web Application Firewall (WAF) docs

@hypercentergr

I have already tried the rate limiting tool with known bots category but it did not work.

May I ask you to share the screenshot of this rule?

@hypercentergr

Any other ideas about the appropriate setup to stop these attacks?

Some are mentioned here (might cost - if you go with Cloudflare Workers):

How to rate-limit facebookexternalhit Requests

ok, something like this: addEventListener('fetch', event => { event.respondWith(handleRequest(event.request)); }) const RATE_LIMIT = 10; // Number of requests const TIME_WINDOW = 60 * 1000; // Time window in milliseconds (1 minute) const BLOCK_DURATION = 5 * 60 * 1000; // Block duration in milliseconds (5 minutes) const USER_AGENT_BLOCKED = 'BlockedUserAgent'; // User-Agent to be rate limited async function handleRequest(request) { const userAgent = request.headers.get('User-Agent') || ''…

Therefrom, you could also spare some CPU load on your origin hosts/server if you’d catch and redirect Facebook’s click links (not cached, but could “cache bust” your server and overwhelm it with requests where each of them need to generate the same webpage HTML document) to your cached page:

Facebook's crawlers - How to rate limit the user agents?

I’ve tracked down quite a lot of requests to my domain with log, therefrom come to the some conclusion for some specific. Despite it might not work for everyone’s case, however I’d like to share my case and example. Name of the Rate Limiting Rule I have: IP Path HR TL-DR; Since Website mostly for Croatia people (10-20k daily visitors), despite it’s a small country, therefore I don’t care for who tries to smuggle some strange requests from the inside country HR since if someone does…