End-to-end Brotli?

Hi, I’ve generated Brotli compressed assets (at -q 11) for all assets on a static SPA, but cannot seem to get the end-to-end Brotli working. When accessing the server directly (Nginx), it correctly respects the Accept-Encoding header, and will properly serve Brotli, Gzip, or no compression.

I’ve read the “All the way to 11” blog (https://blog.cloudflare.com/this-is-brotli-from-origin/) multiple times, and still cannot seem to get this enabled.

I have changed the following settings (and purged the cache, multiple times):

  1. Disabled “Always use HTTPS” (Domain → SSL/TLS → Edge Certificates)
  2. Disabled “Automatic HTTPS Rewrites” (Domain → SSL/TLS → Edge Certificates)
  3. Checked that Mirage is not enabled (as I’m not on a commercial plan, this is not enabled) (Domain → Speed → Optimization → Image Optimization → Mirage)
  4. Enabled Brotli compression (Domain → Speed → Content Optimization → Brotli)
  5. Disabled Early hints (Domain → Speed → Content Optimization → Early Hints)
  6. Disabled Rocket Loader (Domain → Speed → Content Optimization → Rocket Loader)
  7. Disabled HTML Minification (Domain → Speed → Content Optimization → Auto Minify)
  8. Enabled HTTP/2 (Domain → Speed → Protocol Optimization → HTTP/2)
  9. Enabled HTTP/2 to Origin (Domain → Speed → Protocol Optimization → HTTP/2 to Origin)
  10. Disabled “Email Obfuscation” (Domain → Scrape Shield → Email Address Obfuscation)
  11. Disabled “Server-side Excludes” (Domain → Scrape Shield → Server-side Excludes)

I believe this covers all of the items listed in the blog article. If anyone who has this working and knows what settings I have mistakenly made, has any other insight, or knows how I can check this properly, I’d appreciate the feedback. Thanks!

Based on the script here:

https://community.cloudflare.com/t/upcoming-change-to-accept-encoding-header/517115/8

I wrote my own script to get the current settings and then apply the change:

#!/bin/bash

TOKEN=mytoken
[email protected]
ZONE=mydomain.com

set -e

cfdo() {
  local extra=
  if [ ! -z "$3" ]; then
    extra=--data
  fi
  curl \
    -s \
    -X "$1" "https://api.cloudflare.com/client/v4/$2" \
    -H "X-Auth-Email: $EMAIL" \
    -H "X-Auth-Key: $TOKEN" \
    -H "Content-Type: application/json" \
    $extra "$3"
}

ZONE_ID=$(cfdo GET zones|jq -r ".result[]|select(.name==\"$ZONE\")|.id")

echo "ZONE_ID: $ZONE_ID"

declare -A SETTINGS=(
  [email_obfuscation]='{"value": "off"}'
  [rocket_loader]='{"value": "off"}'
  [server_side_exclude]='{"value": "off"}'
  [mirage]='{"value": "off"}'
  [minify]='{"value":{"js":"off","css":"off","html":"off"}}'
  [automatic_https_rewrites]='{"value": "off"}'
)

for SETTING in "${!SETTINGS[@]}"; do
  echo "------------------- $SETTING -------------------"
  echo 'current:'
  cfdo GET "zones/$ZONE_ID/settings/$SETTING"|jq '.result'
  echo -e '\n\nchange:'
  cfdo PATCH "zones/$ZONE_ID/settings/$SETTING" "${SETTINGS[$SETTING]}" | jq '.result'
done

I’ve done this on multiple domains (including a new domain), and I am still not able to get the brotli -q 11 artifacts to be served. Here’s output from one of the domains I applied it to:

$ ./disable-rewrite-configs.sh 
ZONE_ID: <REMOVED>
------------------- email_obfuscation -------------------
current:
{
  "id": "email_obfuscation",
  "value": "off",
  "modified_on": "2023-08-31T08:11:30.596991Z",
  "editable": true
}


change:
{
  "id": "email_obfuscation",
  "value": "off",
  "modified_on": "2023-08-31T08:11:30.596991Z",
  "editable": true
}
------------------- automatic_https_rewrites -------------------
current:
{
  "id": "automatic_https_rewrites",
  "value": "off",
  "modified_on": "2023-08-31T08:06:55.442314Z",
  "editable": true
}


change:
{
  "id": "automatic_https_rewrites",
  "value": "off",
  "modified_on": "2023-08-31T08:06:55.442314Z",
  "editable": true
}
------------------- mirage -------------------
current:
{
  "id": "mirage",
  "value": "off",
  "modified_on": null,
  "editable": false
}


change:
null
------------------- server_side_exclude -------------------
current:
{
  "id": "server_side_exclude",
  "value": "off",
  "modified_on": "2023-08-31T08:11:33.052318Z",
  "editable": true
}


change:
{
  "id": "server_side_exclude",
  "value": "off",
  "modified_on": "2023-08-31T08:11:33.052318Z",
  "editable": true
}
------------------- rocket_loader -------------------
current:
{
  "id": "rocket_loader",
  "value": "off",
  "modified_on": "2023-08-31T23:14:15.812058Z",
  "editable": true
}


change:
{
  "id": "rocket_loader",
  "value": "off",
  "modified_on": "2023-08-31T23:14:15.812058Z",
  "editable": true
}
------------------- minify -------------------
current:
{
  "id": "minify",
  "value": {
    "js": "off",
    "css": "off",
    "html": "off"
  },
  "modified_on": "2023-08-31T23:14:18.654301Z",
  "editable": true
}


change:
{
  "id": "minify",
  "value": {
    "js": "off",
    "css": "off",
    "html": "off"
  },
  "modified_on": "2023-08-31T23:14:18.654301Z",
  "editable": true
}

As shown in the above, the configurations have been applied to my zone.

Cloudflare’s servers always request the gzip'd artifact, regardless. The Nginx config I’m using is:

    brotli off;
    brotli_static on;
    brotli_types *;

    gzip off;
    gzip_static on;
    gzip_vary on;
    gzip_types *;
    gzip_disable "MSIE [1-6]\.";

Nginx is only serving static assets, and I have <name>, <name>.br and <name>.gz all next to each other in the location root. As mentioned previously, I’m able to directly go to the server’s IP and request the assets with different variations on Accept-Encoding headers, and always am served the correct one.

If anyone has insight into this, I’d would really love to know if this is a configuration issue with my server, if it’s because I’m not on a paid plan, or if it’s an issue with Cloudflare’s servers.

The reason this is concerning to us, is about 10 days ago we started having issues with Cloudflare serving serving large WASM files (approximately 15+ megabytes, for a online game). Prior to that, there had been no issue. Specifically: we could no longer download the large WASM artifacts from Cloudflare’s frontend after a new deployment. Basically, we would get “partial transfer” errors, timeouts, etc. when transferring the large WASM. This had not been happening before (and specifically, the WASM has not grown significantly in size).

I believe the issue is that Cloudflare is decompressing/recompressing the WASM and that Cloudflare’s servers are hitting an internal timeout, and are decompressing/recompressing the WASM as per documentation. We’d ideally like to make use of serving Brotli level 11 compressed artifacts, as it shaves off a significant amount of download time, and because of the newly encountered “partial download” errors mentioned above.

Please, if anyone can provide any guidance on this, I would be indebted for life!

Wait a bit before putting more time into this; my understanding is that the rollout has not gone as well as one might have hoped.

Ok, thanks, I appreciate the input.