I know Cloudflare has a lot of genius engineers. Can’t they come up with an algorithm that prevents 404 permalinks/pages from being submitted in Crawler Hints?
One scrapper bot trying to open a page that didn’t exist anywhere on your website, then it will be 404, and then suddenly other legit search engine spiders are also crawling that 404 page.
It becomes an irony when the purpose of Crawler Hints is said to make SE spiders crawl more efficiently and effectively by providing high quality data, but instead, they actually participate in crawling a lot of imaginative pages (404s) generated by other bots.