Cloudflare workers and compressed content at origin

Is it possible to use fetch() to return a response from another site where the data is already gzipped and prevent Cloudflare from decompressing? The origin sets the Content-Encoding header to gzip, and i’m guessing fetch() is inspecting that header and automatically decompressing as a result. The use case is serving a compressed tar archive.

I’ve seen the example here Accessing Compressed Data for Cloudflare Workers Sites - Stack Overflow, but had no success.

If I make a request directly to the origin, the data remains compressed. If i specify -H "Accept-Encoding: gzip" when going through Cloudflare the archive is returned compressed. The data however is returned decompressed when retrieving through Cloudflare and not specifying Accept-Encoding`.

1 Like

(edit.)
If I make a request directly to the origin, the data remains compressed. If i specify -H "Accept-Encoding: gzip" when going through Cloudflare the archive is returned compressed. The data however is returned decompressed when retrieving through Cloudflare and not specifying Accept-Encoding.

Hi @KentonVarda, kindly help our customer here.

Hi @diamon,

I believe this is standard Cloudflare behavior, not specific to workers: If the client states that it does not support gzip (by not sending the Accept-Encoding: gzip header), and the server returns Content-Encoding: gzip, Cloudflare will automatically decompress it. In fact, Cloudflare always sends Accept-Encoding: gzip to the origin server, even if the client did not send this header, and decompresses as necessary.

Normally, this is desirable behavior: It means the server can serve compressed assets without worrying about whether the client supports it, it saves bandwidth between Cloudflare and the origin, it improves cache efficacy (only the compressed version is ever cached), etc.

Your problem here is that you are serving a compressed tarball which you intend for the client to download as-is. The client user-agent isn’t supposed to interpret this file. It’s just downloading.

The right answer here is: Do not send Content-Encoding for this file. Instead, send something like Content-Type: application/tar+gzip – so the compression is described in Content-Type but not in Content-Encoding. This is the usual way to make sure that intermediate proxies and user agents do not try to decompress the data, because this way they see the data as just opaque bytes, not a gzip stream.

Hope that helps!

2 Likes

This topic was automatically closed 24 hours after the last reply. New replies are no longer allowed.