Hi There,
I’m trying to block this crawler / company that uses my content for their paid services.
How do I block w3.howardshome.com from visiting / crawling/ scraping my site?
I’ve searched for IP-adresses, found besides howardshome that they use nameserver.nl nameserver.co and others like thet. I tried various ways of firewall rules and ip acces rules but sofar no luck.
Any suggestions? Thanks in advance!
Best,
Henk
You need to find the network requests in your server log which originated from them. Without that information is is impossible to block anything.
Hi Sandro,
Thanks for you advice. How do I find out about that. My websites are made with www.blogger.com.
On Blogger you do not have access to such information as far as I know.
Does it matter that I pasted an url over it. So for example www.123.com over 123.blogger.blogspot.com?
Any other way?
Over it?
They currently have two IP addresses assigned
What you could do is try and block these addresses on Cloudflare or even their entire address blocks, e.g. 83.98.186.0/24 seems to be assigned to that particular company.
No guarantee either will work or you wont block legitimate requests.
Yes, well the blogger webadress is… http:blogger.blogspot.com or something like that, while you can acces the website by www.websitename.com which is bit nicer and more professional then blogger.blogspot.
Thanks for the IP Adresses, I already found those, and a few others. But perhaps I put it in cloudlflare the wrong way??
I used a firewall rule like this
IP adress / is in / 83.169.0.0/16 (i think this is the nameserver.nl since w3.howardshome.com points to eerste.nameserver.nl
So maybe i should change
Ip adress / equals / * 139 . 162 . 144 . 28
into
ip adress / is in 139.162.144.0/24
Am I doing it then the correct way?
I wanted to add the screen shot to show you what Ive done but it does not seem possible
Can you post screenshots? But it would look all right.
139.162.144.28
83.98.186.81
eerste.nameserver.nl
62.116.159.4
tweede.nameserver.nl
217.160.113.37
83.169.55.4
derde.nameserver.nl
89.146.248.4
nameserver.co
217.160.113.110
83.169.55.50
nameserver.lt
89.146.248.75
nameserver.nl
62.116.159.78
nameserver.tl
74.208.254.74
Range
62.116.0.0/16
83.169.0.0/16
217.160.0.0/16
89.146.0.0/16
These should block it, I’d probably use IP access under https://dash.cloudflare.com/redirect?zone=firewall/tools instead, but these two functions should be equivalent
However if the firewall rules didnt block it, they probably access it from somewhere else.
These last rules wont have any effect.
The ranges above include, both in the first and the last lines, some Europan ISPs, and blocking/challenging them is likely to include legit users in the mix.
You can use a tool such as https://bgp.he.net to check who’s behind each IP and IP range before making a decision about blocking them. Once identifying a hostìng provider, as opposed to an ISP, as the owner of the IP or range, you might prefer to block the AS Number instead of individual IPs, so that if other websites with different IPs under the same ASN are compromised they also will be blocked.
Some of the IP addresses posted along with the ranges do in fact belong to hosting providers. This is a typical sign of a botnet activity, since they use compromised websites to attack other sites, and you can safely challenge these IPs (or their ASN) as they would hardly ever be assigned to legitimate visitors.
I’d somewhat rule that out for the case at hand though. The site in question appears to be relatively legitimate, they even seem to have a /24 network allocated to them by their(?) ISP.
