Cloudflare crawlbot wrong URL 404

Hi,

I notice that lately, the Cloudflare bot tries to crawl my website. However, the crawl URL is wrong hence generating many 404 error logs, ie:

Instead of crawling domain.com/subdomain/folder

it crawls domain.com/folder

I am using a PHP framework.

The crawler is from Cloudflare based on IP address lookup.

Is there a certain setting that I miss?

Please advise.

Thank you

Kind regards,
Ron

have you made any changes to your site’s pages? bots will take time to crawl the new paths.

around 4-5 years ago, before I use Cloudflare, the format was subdomain.domain.com

is this why?

This topic was automatically closed 15 days after the last reply. New replies are no longer allowed.