How to alter the user-agent string

Hi,

Cloudflare’s caching by device type works really well - my problem is that our origin servers don’t always serve the correct version (serving desktop to a mobile device). This isn’t frequent - but happens often enough to become an issue (especially when a wrong version is then stored in the CF cache as ‘mobile’ and then served as such to mobile users until the page expires).

The way I solved this on a previous CDN platform was by appending the word “iPhone” to all requests made to the origin with a header User-Agent matching a certain string:

regex:/(?i)User-Agent:.*(iphone|android.*mobile|BlackBerry|Opera.*Mini|IEMobile|Firefox.*Mobile|Mobile|WebOS|Googlebot-Mobile|rgbmedia-app).*/

I am now trying to achieve the same on the Cloudflare platform, but it doesn’t seem to work. I’m new here so would really appreciate your help.

This is what I did:

I created a new rule called “Mobile user-agent” under Transform Rules → Modify Request Headers:

But this has no effect, it seems. Can you please tell me what I’m doing wrong?

Many thanks,

Bira

I suspect the User-Agent logic on your Origin is different to whatever Cloudflare is using, and this is what is polluting your cache. Because you have enabled Cache By Device Type Cloudflare will append a CF-Device-Type header to each request, which can have three values, mobile , tablet or desktop. So the easiest and most reliable solution would be to change the logic on the Origin to just use the CF-Device-Type header to decide which variant to serve.

If you cannot do that, then there are a few things you can try.

Rather than using your own regex, copy the one from the docs so that your logic matches Cloudflares exactly. As there is a mismatch at present you might end up with the wrong content in each cache shard.

Rather than matching the User-Agent at all, just match against requests where the CF-Device-Type header is one of mobile or tablet.

You should probably also normalise requests with CF-Device-Type equal to Desktop, in case a request that Cloudflare sees as desktop is seen by your Origin as mobile. This means three rules in total will be needed, one for each device type.

Hi Michael,

Many thanks for your detailed reply!

I don’t have a way to use custom header on my origin, unfortunately. I just want requests matching the mobile regex (which, per your advice, I copied from the CF docs) to always have the word “iPhone” in the User-Agent string - either by appending it to the existing User-Agent or by replacing it (preferrably appending, obviously). This is a simple trick that always works on the origin end and has really served me nicely on previous CDNs (EdgeCast, Highwinds and Stackpath) for the past decade.

The “Transform Rules → Modify Request Headers” seems to be the place to do it, but it does absolutely nothing… The user’s original User-Agent header remains unchanged.

Can you give an example of a request, including the headers sent by the client, as well as the headers received by your Origin. (Redact anything sensitive like the IP address)

Yes, thank you.

This is the request headers via developer tools:

GET /writers/bira-goren/ HTTP/2
Host: www.*****.com
User-Agent: Mozilla/5.0 (iPhone; CPU iPhone OS 14_6 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/14.0.3 Mobile/15E148 Safari/604.1
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,*/*;q=0.8
Accept-Language: en-US,en;q=0.7,he;q=0.3
Accept-Encoding: gzip, deflate, br
Connection: keep-alive
Cookie: ***
Upgrade-Insecure-Requests: 1
Sec-Fetch-Dest: document
Sec-Fetch-Mode: navigate
Sec-Fetch-Site: none
Sec-Fetch-User: ?1
Pragma: no-cache
Cache-Control: no-cache
TE: trailers

And this is the request logged on the origin:

[08/May/2023:13:45:17 +0300] "GET /writers/bira-goren/ HTTP/1.1" 404 58164 "-" "Mozilla/5.0 (iPhone; CPU iPhone OS 14_6 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/14.0.3 Mobile/15E148 Safari/604.1"

I botched my reply, so here’s another - and clearer - example:

Request made (as seen in browser)

GET /writers/bira-goren/ HTTP/2
Host: www.***.com
User-Agent: Mozilla/5.0 (Linux; Android 11; SAMSUNG SM-G973U) AppleWebKit/537.36 (KHTML, like Gecko) SamsungBrowser/14.2 Chrome/87.0.4280.141 Mobile Safari/537.36
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,*/*;q=0.8
Accept-Language: en-US,en;q=0.7,he;q=0.3
Accept-Encoding: gzip, deflate, br
Connection: keep-alive
Cookie: ***
Upgrade-Insecure-Requests: 1
Sec-Fetch-Dest: document
Sec-Fetch-Mode: navigate
Sec-Fetch-Site: cross-site
Pragma: no-cache
Cache-Control: no-cache

Reqeust logged on origin:

[08/May/2023:14:04:26 +0300] "GET /writers/bira-goren/ HTTP/1.1" 404 58176 "-" "Mozilla/5.0 (Linux; Android 11; SAMSUNG SM-G973U) AppleWebKit/537.36 (KHTML, like Gecko) SamsungBrowser/14.2 Chrome/87.0.4280.141 Mobile Safari/537.36"

The desired result should have been that on the origin, the user agent would have been the string I put in, which is an iPhone user agent.

Thanks

I added to the origin’s NGINX logs the string $http_cf_device_type - and it returns empty. So not sure what’s going on :frowning: (To be sure, I also added $http_cf_connecting_ip - and this works well).

How did you enable Cache By Device Type? There are a few ways, such as via APO, Cache Rules, Page Rules etc.

In caching → cache rules, In my very first rule (which relates to all site pages) I turned on “Cache by device type”