Currently, there is a limited number of HTTP request headers that you cannot modify. Cloudflare may remove restrictions for some of these HTTP request headers when presented with valid use cases. Create a post in the communityOpen external link for consideration.
This is my request for consideration.
I can’t find the definitive list of all the headers that you cannot modify anywhere, but when I tried creating a request header transform rule to either modify or delete the cache-control request header, I received error messages stating that this header could not be removed or modified.
As per my related post on an APO caching issue, we should really be allowed to control if we want to respect the client’s cache-control request headers. Currently the client can pass a cache-control: no-cache request header and force all request to Cloudflare APO enabled sites to bypass the cache and go straight to the origin server. This is a common request header set in web crawlers like Bingbot. This means that all web crawler traffic ends up bypassing the Cloudflare cache on sites with APO enabled, resulting in tons of unnecessary traffic to origin servers when this is exactly the sort of traffic that the Cloudflare cache should be shielding a server from.
Being able to modify or remove this request header in a transform rule, possibly in conjunction with a bot score check, user-agent check, etc., would hopefully fix this issue with web crawlers bypassing the cache on Cloudflare APO websites.
It needs to happen before APO runs. I’m pretty sure APO runs in the Workers stage. That list you show is just the order of operations that rules are applied to the request, before the requested content is retrieved from cache or pulled from origin I believe.
If Cache Rules was the place the value was actually retrieved from the cache, then things like firewall rules wouldn’t apply to cached content, which doesn’t sound correct at all.
Then let’s let a Cloudflare employee chime in on exactly where APO falls into that list if you don’t know for sure.
The APO issue I’m facing is something that Cloudflare employees on this support site, (as well as the APO documentation), have said is working as intended. I’ve fixed many other issues I have with APO via page rules and URL rewrite rules, so I think it is perfectly reasonable to think I could also fix some of the issues I have with APO using request header transform rules. APO works fine with the various types of Cloudflare rules, so I’m not sure why you think this would have to be fixed by a change to APO instead of Cloudflare simply giving people the option to change this behavior via a transform rule. Even the currently recommended fix for this issue is done via page rules, I’m just suggesting an alternative fix via request header transform rules that would be cleaner and have less side-effects.
The link in the official Cloudflare Transform Rule documentation said to post HTTP headers on this discussion forum for consideration, and to include a use case. That is what I have done. There are other use cases for including Cache-Control request header support in transform rules. I think the main use case would be if you were using a second-level cache on your origin server, and you wanted to prevent Cache-Control: no-cache headers from making it to your origin server and possibly bypassing your second-level cache.
If Cloudflare totally ignores Cache-Control: no-cache request headers (outside of APO) then I don’t see any harm in them allowing us to change that request header. My guess is that header is blocked because they don’t want us changing it in the response headers, but they should be able to enable it in request header transform rules with no issue.
No, I’m still trying to get someone at Cloudflare to look at this.
My issue is specifically for request headers sent by the client. Your issue is response headers sent by the origin server. You can create page rules in Cloudflare to ignore/override the server’s cache setting. That would be the Edge Cache TTL and Browser Cache TTL settings in page rules.