Bot user is continuously pushing server to serve pagenotfound.html file

IF the bot user is continuosly served with page not found pages(after many redirects) with error code 404/403 ,how to identify the pattern of such bot users and rate limit them?

Can you post a log excerpt?

I don’t have any logs . Any ‘page not found’ exceptions are actually handled via our legacy server side only.

Without any information it is difficult to detect a pattern.

you need access logs?

For example.

E.g. is it always the same IP address or block?

yes.But I agree ,it is possible via Cloudflare to do IP ratelimiting.
But ,I am concerned about bot user trying with new IPs .(say,assumption)
This could lead to DDOS attacks.

Well, if you have genuinely looking requests to / each from distinct addresses there is little you can do except for hoping Cloudflare has information for these addresses based on which requests could be blocked. That would be on one hand increasing your security level, on the other evaluating cf.client.bot in a firewall rule.

As far as rate limiting is concerned you should check out https://support.cloudflare.com/hc/en-us/articles/115004256008

It really comes down to the attack vector.

cf.client.bot - Can you please explain on this?

Plus cf.threat_score

Thanks a lot.But,how to define rule which has ‘cf.threat_Score’ more than 10.IS it a page rule or firewall rule?Do you have any sample webpage which does the setting as you advised.?

A firewall rule with cf.threat_score > 10 should do the trick.

Thanks again.Is this firewall rule a new beta feature in Cloudflare?

Its a new feature, but not beta any more.

Thanks again…

This topic was automatically closed after 14 days. New replies are no longer allowed.