Cloudflare is killing my website

so i found there is more than 100k link is pending for indexing all conatins cloudfalre scripts
example of link:
site.com/cdn-cgi/challenge-platform/h/g/scripts/cb/invisible.js?cb=760d6024a8945872

It sounds like you don’t have a robots.txt setup or anything to tell search engines not to index those URLs.

I would recommend you add a robots.txt to your website to exclude URLs you do not want indexed, such as those under /cdn-cgi.

If you’re talking about some other kind of indexing, then I would recommend you exclude *.js files from that.

i do not know much about robots.txt file any guide please

How Google Interprets the robots.txt Specification  |  Documentation  |  Google Developers is a good reference.

Can you provide some more information about exactly what you mean by “killing [your] website” though? Where are you seeing this? What is pending indexing? Can you share any reports/screenshots?

1 Like

it was pending for indexing in google console
can check if this robots txt is good

User-agent: *
Disallow: /wp-admin/
Allow: /wp-admin/admin-ajax.php
Disallow: /cdn-cgi/

This topic was automatically closed 15 days after the last reply. New replies are no longer allowed.