So, there is a known Content Scraper out there. I know the website, but they use cloudflare. I use cloudflare too.
What I’m worried about is more stopping them from scraping more content from my site than taking the stuff down.
I guess my question is:
Does the firewall protect sites from other sites that use Cloudflare?
Does Cloudflare unintentionally mask sites that use bots with their services?
Cloudflare masks the IP of it’s user’s websites, which is great, but it isn’t said how that can be used maliciously, and if so, how other cloudflare users are protected.
It’s not that I don’t love the service that Cloudflare provides. I do. But lately, I’ve realized that there is a lot of hits to the server when not many people are on it at the time to drain that set of resources. And I don’t want to use rate limiting. The site uses a lot of requests, and I’m worried about the end of the month bill, especially considering there is no guarantee that it will stop that particular scraper.
But considering I know what site it is, why is it so hard to protect yourself against someone like that? Things shouldn’t be so hard.