Hi, everybody!
I have a question. Let’s see if anyone has already done so. I explain: I have a page with many URLs, all have Amazon products on it. In fact, when loading each of these pages a request is generated against Amazon (via API) which is responsible for displaying the updated products.
My problem is that, as I was saying, I have many URLs and every time the google robot, yahoo, bing… etc… sends a request to the API and it ends up not working (the available requests are finished).
The question is: Is there any way to cache the amazon products table with cloudflare so that every time the spiders pass by, my Amazon API does not stop working?
Thanks in advance
Without seeing the actual page, it’s difficult to give safe advice.
Other than the product listing, is there anything on that page that’s tied to a user session? Like a shopping cart?
My suspicion is that this is a shopping site and it’s not save to cache that page. I would hope that whatever the site is built with would have a way to reduce these API calls by caching the results.
Thanks for answering, sdayman.
My website is https://migato.online …unfortunately at the moment it does not show Amazon products because of the problem mentioned above. It should be displayed just like the website “Lavadoras LG ✅ Lavadora.info”
This website does show the products because the Amazon API it works with is different and more powerful (they only give it to you if you sell many products) and it responds to a greater number of requests so it does not have problems responding to spiders. In addition, this washing machine website has a lower number of URLs.
The keyword sent to Amazon is what is behind the slash (/) in this case LG.
The site does not have a cart, it sends you to amazon to make your purchase from there. Typical Amazon affiliate site.
Thank you very much for your help.
The good news is those are all affiliate links to Amazon, so nothing on your site changes for user interaction. How about trying this with a Page Rule:
Match: migato.online/
Settings: Cache Level (Cache Everything), Edge Cache TTL (24 hours).
This will just cache the home page as a test. Hopefully that will stop the API from hitting Amazon to keep populating your item list. And save that for 24 hours.
As a followup question, does your site have admin pages or any type of login? You don’t need to divulge the URL. Just a Yes or No answer for reference.
Thank you very much. I’m going to try, though I don’t have much faith.
Right now the products are shown, it seems that the amazon API has been renewed.
the ideal would be to cache everything now… or cache in the future only if the products appear, that is to say, to make an intelligent caching that does not cache if the products do not appear because when a google or yahoo robot passes by and makes more than 8500 requests (it has to do 15K to complete the entire web), the API will be exhausted and the products will be left to be shown.
The website does not have an administrator login or similar.
Thank you!
Do you know what the API rate limit is? You may be able to slow down Google’s crawl rate.
if I’m not mistaken, that’s 8500 requests a day. How do you reduce the tracking speed?
I’ve reduced google crawl time on search console. This was 3 days ago and now it shows products when the page is refreshed several times… now the question is: google robots are not the only ones that go through the websites, there will be many other search engines that do it, contributing to the exhaustion of the API.
Thanks for the help. I still need, when my website is showing all the products, to “take that picture” of it with cloudflare.
This will cache the home page.
You didn’t answer my other question, so I can’t suggest a safe caching rule for your product pages.
I’ve already tried that rule, so far it hasn’t had any positive results.It is possible that cloudflare is caching when the products do not appear. But that’s not a problem of the cache, but that the products, at the time of the cache, were not shown on the web.
Regarding this, as I already told you, no: The website does not have an administrator login or similar.
I hope I explained myself well but and first of all, excuse my English
Thank you very much for your help!
Sorry, I see that response now. So it’s safe to cache the entire site.
I checked your home page and that page rule is not taking effect. Let’s try again:
Page Rule #1
Match: migato.online/*
Settings: cache level (everything), edge cache TTL (24 hours)
Please post a screenshot of your Page Rules page after you add the rule.
Thanks, I appreciate your help.
Since that rule didn’t work. I’m trying out for the home page one rule and for the rest of the page another. I’ve changed the settings to 4 days…
You did not add the Cache Everything setting for the Cache Level. You need both Edge Cache and Cache Everything.
1 Like
That did it. You shouldn’t need that second rule. The first rule should match everything.
1 Like
Perfect. Let’s see if we get lucky!! Thanks, I’ll keep reporting.
1 Like