Cloudflare is killing my website

Website, Application, Performance
1

so i found there is more than 100k link is pending for indexing all conatins cloudfalre scripts
example of link:
site.com/cdn-cgi/challenge-platform/h/g/scripts/cb/invisible.js?cb=760d6024a8945872

2

It sounds like you don’t have a robots.txt setup or anything to tell search engines not to index those URLs.

I would recommend you add a robots.txt to your website to exclude URLs you do not want indexed, such as those under /cdn-cgi.

If you’re talking about some other kind of indexing, then I would recommend you exclude *.js files from that.

3

i do not know much about robots.txt file any guide please

4

āļ§āļīāļ˜āļĩāļ—āļĩāđˆ Google āļ•āļĩāļ„āļ§āļēāļĄāļ‚āđ‰āļ­āļāđāļēāļŦāļ™āļ”āļ‚āļ­āļ‡ robots.txt  |  āđ€āļ­āļāļŠāļēāļĢāļ›āļĢāļ°āļāļ­āļš  |  Google Developers is a good reference.

Can you provide some more information about exactly what you mean by “killing [your] website” though? Where are you seeing this? What is pending indexing? Can you share any reports/screenshots?

1 Like
5

it was pending for indexing in google console
can check if this robots txt is good

User-agent: *
Disallow: /wp-admin/
Allow: /wp-admin/admin-ajax.php
Disallow: /cdn-cgi/

update this solved my problem

6

This topic was automatically closed 15 days after the last reply. New replies are no longer allowed.