Crawler Switch for Loadbalancing


Planning to use Cloudflare Traffic LB to send some traffic to a new frontend which has the same content than the old one but the source code differs since it is a complete new template engine behind.
Now in terms of SEO/Crawlers - is there any possibility to create a crawling switch? Based on user-agent for example? So I could define a rule to send all traffic coming from the google bot to a specific system/frontend? Basically this would help to send the crawler only to one frontend to prevent the crawlers from indexing to different source codes.


This topic was automatically closed 5 days after the last reply. New replies are no longer allowed.