Transform Rules cannot set Authorization header?

I am confused by this error message. What is the point of the feature if arbitrary headers are locked from modification?

image

This limitation is not mentioned in the documentation. Is it merely a bug?

1 Like

Hi @zeblote,

My guess would be that it is intentional, based on the error received. Tagging @smarsh to see if an explanation can be given and if the docs could be updated to reflect this.

Well instead of updating the documentation, it would be much better to remove this limitation. Here’s why:

This is the only missing piece to being able to expose private buckets from many cloud storage vendors behind Cloudflare, now that Transform Rules feature exists.

Many cloud storage providers give you the ability to create private buckets that can only be accessed using an Authorization header with some token.

Now, you could theoretically give the token to users so they can include the correct Authorization header with their request, but this is generally a terrible idea. They would then also be able to request files from the bucket directly, bypassing Cloudflare cache, and causing undesired costs for you with their downloads. This would be equivalent to giving someone your origin IP behind Cloudflare.

A much better method is to use the new Transform Rules to inject this Authorization header for the paths that are routed to the cloud storage provider. This way, it is only ever sent between Cloudflare and the origin, and never exposed to the user.

I cannot see any potential security problems if you allowed to modify this header. After all, you’re just causing it to send some value you set to your own origin. That’s completely harmless, but extremely useful in many cases like the above. So, please consider allowing Transform Rules to set the Authorization header.

I’ve raised this with the team, we’ll review and see if we can remove the restriction on this header.

As a rule of thumb, we erred on the side of caution and started off restrictive with which header names could be modified, with a view to addressing individual requests (like ‘Authorization’) on a case by case basis. Some headers we wont allow to be modified for security reasons, however we’ve already derestricted a number of header names after request.

I’ll keep you posted. Thanks again for raising, this is a great use case.

4 Likes

Great to hear, thanks!

As for potential security issues, what could there really be? It’s your own origin that the header is sent to. So even if this gives you the ability to make requests pretend to be a specific user, it’s still your own site, so you could have already done that anyway. Or if you really wanted to, you could use Workers to mess with everything. But that incurs extra costs to put before cached static files. I see only good things coming if this header is allowed to be set.

Just to chime in, I too would be interested in adding the Authorization header specifically to be able to authenticate to an upstream cloud storage provider. Indeed one can also do this with a worker, but it would be nice to not eat into those usage limits.

Thanks,

Hey @smarsh , just wondering if you have any updates on this!

Glad to see other people are also interested in this header. It’d be incredibly useful, if not required, for some of our upcoming features.

Hey. We’ve got all the code merged and we’re in the process of releasing. Expectation is this should be globally deployed and available by late next week. I’d say give it a try on Thursday 15th :slight_smile:

4 Likes

Awesome!

Hey @smarsh while we wait, I have a related question. Is the value of the Authorization header included in the cache key? It would be absolutely amazing if not, because then we can rotate out tokens for storage providers that only generate time-limited ones WITHOUT losing any cache hits.

1 Like

Supposedly not, unless someone defined it under Page Rule - Custom Cache Key setting (only available in Enterprise plan)

3 Likes

Fantastic! I can’t wait to test this.

And it’s live! I made a scheduled worker script to obtain a new download authorization and then replace the transform rule every day and it works perfectly fine without causing any cache misses.

1 Like

That said @smarsh, it would be very useful if transform rules could be updated from API using a specialized API token. Currently it requires giving the worker global API key which is not very nice.

1 Like

How did you know? I am trying to collect a full list of all headers that affect the cache key, but am having trouble finding accurate and consistent information. So far I seem to only find random blog posts that contain hints at some headers that are included:

The goal being, obviously, to kill any unnecessary header that can influence the default cache key using transform rules:

image

Don’t want anyone having the ability to bypass my cache on real urls.

We hear you. Its already on our backlog for the coming months :slight_smile: Along with a dedicated dashboard ‘permission’ so customers can create users who can ‘just’ modify Transform Rules. Today its shared with Firewall Rules.

Lots of incremental changes coming, all of it good :slight_smile:

2 Likes

That’s what I thought, but when I tried creating a token with access to edit the zone “firewall services” it said “authentication error” trying to read /zones/id/rulesets or set /zones/id/rulesets/id.

It seems to me that this API only works with the global API key. If not, what exact permission(s) do I need to grant on the token?

It is global API key only at the moment unfortunately.

1 Like