503 errors with Ahrefs

What is the name of the domain?

yoursafety.training

What is the error number?

503

What is the error message?

Status HTTP/2 503 Date Tue, 07 Jan 2025 12:42:26 GMT Content-type text/html; charset=UTF-8 Cache-control private, max-age=0, no-store, no-cache, must-revalidate, post-check=0, pre-check=0 Expires Thu, 01 Jan 1970 00:00:01 GMT Referer-policy same-origin Tiger-protect-security Tiger Protect : règles de sécurité web Cf-cache-status BYPASS Report-to {“endpoints”:[{“url”:“https://a.nel.cloudflare.com/report/v4?s=YKZ5b51qTPuM8YJ7disae2vdoELCzZF7SiUDHXJ53LeQi5YogW1lxUd2%2BCQ5ozYyB%2BZ8yLixOQ%2FVSYj47HRtw%2F2HeXkonoV1m%2BRvH0ASVngZFc3OioG0O3zJJy1aYWLVTwEDCG12”}],“group”:“cf-nel”,“max_age”:604800} Nel {“success_fraction”:0,“report_to”:“cf-nel”,“max_age”:604800} Vary Accept-Encoding Server cloudflare Cf-ray 8fe407192a44389a-IAD Alt-svc h3=“:443”; ma=86400 Server-timing cfL4;desc=“?proto=TCP&rtt=591&min_rtt=581&rtt_var=185&sent=6&recv=8&lost=0&retrans=0&sent_bytes=3426&recv_bytes=892&delivery_rate=4967409&cwnd=252&unsent_bytes=0&cid=6c7b7ad2b63312c5&ts=293&x=0”

What is the issue you’re encountering

Ahrefs suddenly reports many 503 errors for pages that actually work

What steps have you taken to resolve the issue?

I have:
Hi everybody,

  • read all posts to find a fix
  • disabled Speed Brain Speed Brain and Hearly Hints.
  • allowlisted the new IP range of Ahrefs’ bots
  • slown down the crawl speed (15 pages per minute)

Thanks

What are the steps to reproduce the issue?

Run another Ahrefs’ crawl…

Screenshot of the error

Looking at the screenshot and seeing for the first time the HTTP header Tiger-protect-security, wonder if the origin host security has something to do about it :thinking:

Have you tried reaching out to your hosting/service provider in the meantime?

Does it work fine over HTTPS when Cloudflare is temporarly paused? :thinking:

  • Use the “Pause Cloudflare on Site” option from the Overview tab for your domain at dash.cloudflare.com .
  • The link is in the lower right corner of that page.

Kindly, see post below with an idea for finding a solution:

Many thanks Fritex! I will give it a try tomorrow.

Hi Fritex,

After different checks and purges (Litespeed WP plugin, REDIS, Cloudflare) I keep on having 503 errors on Ahrefs. So I have tried a similar site audit on Semrush and… problems are gone!

This is another topic, but why (and where) would Ahrefs keep cached information for Site Audit?

Many thanks for your time and support.

Jerome

Thank you for feedback.

Ou wow! Multiple cache systems here :thinking:

I guess, at their servers? :thinking:
Or if not, something served to them might be left cached.

I am not quite sure, am not using neither of those services and don’t know more in details how do they operate :man_shrugging: