Upcoming Change To Accept Encoding Header

[mbullock]

Hi Cloudflare Community!

I wanted to inform you of an exciting upcoming change to Cloudflare’s Accept-Encoding header. This change will enable Cloudflare to support the transmission of Brotli compressed data from your origin servers.

Currently, Cloudflare forwards all HTTP requests to the origin with the following header:
Accept-Encoding: gzip

Over the coming weeks, we will be transitioning to the following header to allow the origin to send Brotli compressed content to Cloudflare:
Accept-Encoding: gzip, br

Impact of the Change

No action should be required. If you wish to disable Brotli at your origin or do not have Brotli support enabled, Cloudflare will continue to support Gzip and uncompressed content as usual.

If your origin supports Brotli encoding, you will be able to take advantage of its high compression ratios when transferring data to Cloudflare.

What is Brotli?

Brotli is a highly efficient data compression algorithm developed by Google. It offers high compression ratios and fast decompression speeds, making it particularly suitable for compressing text-based files such as HTML, CSS, and JavaScript. Since 2019, Brotli has been widely supported by web browsers. Cloudflare has been offering Brotli compression to end users customers since 2017, resulting in improved website performance and smaller file sizes. We are now extending support to origin servers.

Thanks,

Matt Bullock
FL Product Manager

[cherryjimbo]

This is fantastic!!

Will Cloudflare still decompress and then recompress when serving that data to eyeballs? For example, if I serve a highly compressed JS file with Brotli to Cloudflare, will my highly compressed version also be delivered to the user, or will Cloudflare decompress and the recompress at a lower compression ratio?

[sandro]

I’d assume they have to decompress the file, in order to apply the usual proxy features. Plus, they need to take the acceptable client encoding into account.

[mabba]

@cherryjimbo we won’t unless, as @sandro mentioned, we need to apply proxy features. That’s it, if you disable all HTML-based features (ex: email obfuscation) and send us brotli-compressed HTML, we will keep the same compression strategy you picked!

[sandro]

Does this mean Cloudflare would cache based on the compression? If I send br compressed data and Cloudflare caches that, that would not work with a client which does not support br.

[mabba]

this behavior won’t change from the one you can currently use. Right now, if client is not requesting brotli, we will not serve them brotli-compressed responses.

[sandro]

Yeah I assume it wouldn’t change from the current setup, just wondering about Cloudflare keeping the compression.

I would imagine that only works for content where Cloudflare was instructed not to cache, the proxy is not doing anything, and when the client accepted br as well, right?

[mbullock]

To maintain the same compression as what the origin sends (i.e., Brotli Compression Level 11), the following proxy features should be disabled:

• Email Obfuscation
• Rocket Loader
• Server Side Excludes (SSE)
• Mirage
• HTML Minification - JS and CSS can be left enabled.
• Automatic HTTPS Rewrites (Not to be confused with always use HTTPS)

The API calls / BASH Script can be found below. Note that the below will disable all Minification features (HTML, CSS, JS), with only HTML requiring to be disabled.

If any of these features are enabled, your origin can still send Brotli Compression level 11. However, we will decompress, apply the feature(s), and recompress on the fly.

For browsers that do not accept Brotli Compression, we will decompress and send GZIP Compressed or Uncompressed.

#!/bin/bash

# Cloudflare API credentials
api_token="R3d4ct5dXXXXXXX"
email="[email protected]"
zone_id="c0779e7XXXXXXXXXXXXXXX6975e2fb9b"


# Disable Email Obfuscation
curl -s -X PATCH "https://api.cloudflare.com/client/v4/zones/$zone_id/settings/email_obfuscation" \
  -H "X-Auth-Email: $email" \
  -H "X-Auth-Key: $api_token" \
  -H "Content-Type: application/json" \
  --data '{"value": "off"}'

# Disable Rocket Loader
curl -s -X PATCH "https://api.cloudflare.com/client/v4/zones/$zone_id/settings/rocket_loader" \
  -H "X-Auth-Email: $email" \
  -H "X-Auth-Key: $api_token" \
  -H "Content-Type: application/json" \
  --data '{"value": "off"}'

# Disable Server Side Excludes (SSE)
curl -s -X PATCH "https://api.cloudflare.com/client/v4/zones/$zone_id/settings/server_side_exclude" \
  -H "X-Auth-Email: $email" \
  -H "X-Auth-Key: $api_token" \
  -H "Content-Type: application/json" \
  --data '{"value": "off"}'

# Disable Mirage
curl -s -X PATCH "https://api.cloudflare.com/client/v4/zones/$zone_id/settings/mirage" \
  -H "X-Auth-Email: $email" \
  -H "X-Auth-Key: $api_token" \
  -H "Content-Type: application/json" \
  --data '{"value": "off"}'

# Disable HTML Minification NOTE: you only need to disable HTML this call disables all
curl -s -X PATCH "https://api.cloudflare.com/client/v4/zones/$zone_id/settings/minify" \
  -H "X-Auth-Email: $email" \
  -H "X-Auth-Key: $api_token" \
  -H "Content-Type: application/json" \
  --data '{"value":{"js":"off","css":"off","html":"off"}}'

# Disable Automatic HTTPS Rewrites - NOT to be confused with Always use HTTPS
curl -s -X PATCH "https://api.cloudflare.com/client/v4/zones/$zone_id/settings/automatic_https_rewrites" \
  -H "X-Auth-Email: $email" \
  -H "X-Auth-Key: $api_token" \
  -H "Content-Type: application/json" \
  --data '{"value": "off"}'

[sandro]

All right, so essentially one only needs to disable all features which possibly change the content. Caching itself is fair and Cloudflare will automatically recompress as gz if a client requested a br cached file but does not support br.