Amazon Price scraping - No Like


Amazon is price scraping our site and we like to block them. We run PRO level and the setting Super Bot Fight Mode, but this does not help.

Any tips how to solve it?

Best Magnus


I am sorry to hear you have an bad experience with Amazon scraping :open_mouth:

Nevertheless, thank you for asking. I believe if you are using Cloudflare for your domain, there are multiple ways how could we detect and block “bad guys” :wink:

Great, awesome! Pro plan offers “Web Application Firewall” (Managed WAF Rules), “Cloudflare Managed Ruleset” and “Package: OWASP ModSecurity Core Rule Set” which we can enable with a single click and configure per demand.

Despite of this, Pro plan has got an option to “Configure Super Bot Fight Mode” to challenge “Definitely automated” bots.

Furthermore, you can use Firewall Rules to block Amazon AS numbers - there are multiple as far as I remember - and even more like old HTTP/1.0 requests, or block some known “bad” user-agents, crawlers, etc.

Under the Firewall Setings, you can set the Security Level, Challenge Passage, enable Browser Integrity Check and few others.

You could also track your Firewall Events and check for any suspicious or being challenged or blocked, that way you can tune-up and adapt your security settings as you need.

Using Cloudflare Scrape Shield, you can protect written e-mail addresses in your HTML content of a webpage by enabling Email Address Obfuscation for example.

I am sharing my settings at the below screenshots from which you can compare with yours and analyze :

In terms of Amazon, in my case and my example I am blocking the three “Amazon” ASN numbers with Firewall → IP Access Rules as follows:


Wow, Thanks a lot. We try it. Cheers Magnus

And thanks again. But sorry to say the settings did not work. Amazon is still scraping the prices.
Do you know any other ways to stop Amazon price scraping?
Best Magnus

Can you see the requests in your web server logs? If so, can you share one of the log entries?

1 Like

Hi and thanks

I will comeback with it.

Best Magnus


We having a lot of promblem with Amazon Price scraping and have done changes and blocked IP.

Is there any one that know how to solve it.

Best Magnus

There are some discussions regarding on how to deal with bots:


Yeah, given how complex nowadays bots are, it’s doubtful that any non-enterprise solution can keep up with detections right off the bat.
Amazon is an excellent example of an arms races fight. Amazon dedicates millions to keeping their scrapers up to date to serve the lowest prices across the web; it’s a bit unfair to expect any $20-200 solution to be bulletproof against that.

My best bet to counter Amazon (without prior experience in doing so) would be to block their ASN completely; however, it wouldn’t surprise me if they used more than one ASN to scrape prices.


This topic was automatically closed 15 days after the last reply. New replies are no longer allowed.