Always online - aggressive crawling?


#1

Hi

The Cloudflare-Alwaysonline crawler seems to excessively hit our frontpage from time to time, creating sort of a mini DDOS attack against the backend server. (400+ request within a minute)

Is anyone else experiencing this?


#2

Can you confirm the IP address in your log is one of the CF IPs? The user agent might be spoofed.


#3

We didn’t expose the X-Forwarded-for in our log so I only see the ip-adress of the amazon loadbalancer. :frowning:
We’ve changed the log-format so we can check the IP-adress next time it happens. We have, however, decided to turn off the “Always online” feature as a preemptive step.


closed #4

This topic was automatically closed after 14 days. New replies are no longer allowed.