Cloudflare-Smart-Transit/1.0 User-Agent requests

For the last month, I’ve noticed some requests with this format in the log of my web server behind Cloudflare: - - [06/May/2017:21:40:28 +0000] “GET / HTTP/1.1” 301 0 “” “Mozilla/5.0 (compatible; Cloudflare-Smart-Transit/1.0; +” 0/159085 - - [06/May/2017:21:40:29 +0000] “GET / HTTP/1.1” 301 0 “” “Mozilla/5.0 (compatible; Cloudflare-Smart-Transit/1.0; +” 0/168656 - - [06/May/2017:21:40:30 +0000] “GET / HTTP/1.1” 301 0 “” “Mozilla/5.0 (compatible; Cloudflare-Smart-Transit/1.0; +” 0/205860

There are always 3 at a time, with an interval of between 20 and 50 minutes between every 3 requests, and they all come from the Cloudflare IP range.
The only HTTP headers provided are:
User-Agent: Mozilla/5.0 (compatible; Cloudflare-Smart-Transit/1.0; +
Accept-Encoding: gzip

The only place I’ve found that User-Agent menctioned is in this CS stats site.
They are answered with a 301 redirect because I redirect every request that does not have my site as the Host header.

Should I be concerned about these requests? Am I breaking any Cloudflare functionality redirecting them?

1 Like

Typo: the Host header is my origin server IP, which I prefer to not disclose.

Clarification: the 301 redirects to the correct domain.

I’m not a Cloudflare support member, just a user like you.
It seems you are redirecting a Cloudflare’s crawler and blocking whatever the purpose is.

That user-agent, specifically when matching this exact string:
Mozilla/5.0 (compatible; Cloudflare-Smart-Transit/1.0; +

When originating from one of Cloudflare IPs:

Is part of a new tool/product that monitors network health (between Cloudflare PoPs, and also between Cloudflare and origins). This data is then used by to dynamically update the routes that traffic takes which provides both a smart transit capability as well as tiered caching.

There is no downside to this operating (the product announcement will be in the near future and the improvements as a result of this are really impressive) but you may wish to have your origin web server return a static file for this user agent if not doing so is resulting in additional computation by your web app (i.e. if the network monitor hits a page that will do database lookups, since the tool only cares about measuring the network rather than the page speed, perhaps you want to avoid those DB queries).



Great! Thanks for the clear explanation.

How to avoid this hit after disable Cloudflare services, I still got thousand traffic in my homepage url.