Confirming brotli connectivity

Website, Application, Performance
1

What is the name of the domain?

What is the issue you’re encountering

I’d like to take advantage of broli compression from end-to-end. How can I confirm it’s working?

What steps have you taken to resolve the issue?

I’ve confirmed our website (https://linuxsecurity.com) supports brotli compression, but I’m not sure if cloudflare is then passing compressed files onto the browser. Here’s what I see from curl:

$ curl --head --header 'Accept-Encoding: br' --silent https://linuxsecurity.com
HTTP/2 200
date: Tue, 03 Sep 2024 14:39:55 GMT
content-type: text/html; charset=utf-8
expires: Wed, 17 Aug 2005 00:00:00 GMT
cache-control: max-age=120, no-transform
pragma: no-cache
vary: Accept-Encoding,User-Agent,Origin
x-content-encoded-by: Joomla
access-control-allow-origin: *
access-control-allow-methods: POST, GET, OPTIONS, DELETE, PUT
access-control-allow-headers: x-requested-with, Content-Type, origin, authorization, accept, client-security-token
access-control-expose-headers: Content-Security-Policy, Location
access-control-max-age: 600
strict-transport-security: max-age=31536000; includeSubDomains; preload
permissions-policy: accelerometer=(), autoplay=(), camera=(), cross-origin-isolated=(), display-capture=(), encrypted-media=(), fullscreen=(), geolocation=(), gyroscope=(), keyboard-map=(), magnetometer=(), microphone=(), midi=(), payment=(), picture-in-picture=(), publickey-credentials-get=(), screen-wake-lock=(), sync-xhr=(), usb=(), xr-spatial-tracking=(), clipboard-read=(), clipboard-write=(), gamepad=(), hid=(), idle-detection=(), serial=(), window-placement=()
last-modified: Tue, 03 Sep 2024 13:53:48 GMT
etag: W/"57d6466c3a32238efb52b396d96e2bbe"
accept-encoding: br
cf-cache-status: HIT
age: 2650
report-to: {"endpoints":[{"url":"https:\/\/a.nel.cloudflare.com\/report\/v4?s=iedJT6tGxaN%2FnnCWFQDC8CdXW%2BrfymRvNG77Th7mlKTOUgZDZecwImRhIxM0kcY83LAvMKhcHrGjpa4HkiR9r%2FxiHLfREZaJNWU2gSGkotgSYt0mpWwLtJDAa5NjgtX8w5Uj"}],"group":"cf-nel","max_age":604800}
nel: {"success_fraction":0,"report_to":"cf-nel","max_age":604800}
expect-ct: max-age=86400, enforce
referrer-policy: same-origin
x-content-type-options: nosniff
x-frame-options: SAMEORIGIN
x-xss-protection: 1; mode=block
server: cloudflare
cf-ray: 8bd67df11c507cb1-EWR

Would there be much of a benefit to compressing the individuals manually and delivering them instead of having the stream compressed?

2

You would do something like curl -s -I -H 'Accept-Encoding: br' https://linuxsecurity.com/ | grep content-encoding. I tried, and it did not appear to be on for your site. I would check your backend to make sure that supports broli.

It makes the files smaller so easier to store and send.

3

Yes, that is exactly what I did to produce the above results, but I thought it was the “accept-encoding” parameter that I was looking for, given that’s the header I’m including with my curl command.

I understood that content-encoding was for compression between the connection between the server and cloudflare, and accept-encoding was for compression between cloudflare and the browser?

What happened to the brotli setting in Cloudflare? Is it now enabled by default? Where is this controlled in cloudflare?

4

You can check via web site tool HTTP Header Checker - Check HTTP Response Headers With curl | KeyCDN Tools

for your site I see you only have gzip compression enabled content-encoding: gzip according to KeyCDN curl header test tool

HTTP/2
200
date: Wed, 04 Sep 2024 00:55:11 GMT
content-type: text/html; charset=utf-8
expires: Wed, 17 Aug 2005 00:00:00 GMT
cache-control: max-age=120, no-transform
pragma: no-cache
content-encoding: gzip

for curl checks make sure curl does have brotli support first.

curl -V
curl 8.9.1 (x86_64-pc-linux-gnu) libcurl/8.9.1 quictls/3.1.5 zlib/1.2.13 brotli/1.1.0 zstd/1.4.4 libidn2/2.3.7 libpsl/0.21.5 libssh2/1.11.0 nghttp2/1.62.1 ngtcp2/1.6.0 nghttp3/1.4.0 OpenLDAP/2.6.8
Release-Date: [unreleased]
Protocols: dict file ftp ftps gopher gophers http https imap imaps ipfs ipns ldap ldaps mqtt pop3 pop3s rtsp scp sftp smb smbs smtp smtps telnet tftp ws wss
Features: alt-svc AsynchDNS brotli GSS-API HSTS HTTP2 HTTP3 HTTPS-proxy IDN IPv6 Kerberos Largefile libz NTLM PSL SPNEGO SSL threadsafe TLS-SRP UnixSockets zstd

example testing Cloudflare site shows content-encoding: br brotli supported/compressed that is what you should be looking for and not HTTP response with accept-encoding

curl -I -H "Accept-Encoding: br" https://www.cloudflare.com
HTTP/2 200 
date: Wed, 04 Sep 2024 00:53:09 GMT
content-type: text/html; charset=utf-8
cache-control: public, max-age=0, must-revalidate
strict-transport-security: max-age=31536000; includeSubDomains
permissions-policy: geolocation=(), camera=(), microphone=()
referrer-policy: strict-origin-when-cross-origin
x-content-type-options: nosniff
x-frame-options: SAMEORIGIN
x-gww-loc: EN-US
x-pgs-loc: EN-US
x-rm: GW
x-xss-protection: 1; mode=block
set-cookie: __cf_bm=XDU8P0p_mHoc4vJaesZxOwyMXu0Rw8AXiWIwH.lWiBs-1725411189-1.0.1.1-35p4IWGf9iGpIMF9sZgvPTRwUWxyHU.1VKBdL1SUc8cEX20eDiAkfBvgTjfM94eSN9G_ZRrbBlVMnr16R40XzGWUFK8TErStoYAo5ChDXBE; path=/; expires=Wed, 04-Sep-24 01:23:09 GMT; domain=.www.cloudflare.com; HttpOnly; Secure; SameSite=None
report-to: {"endpoints":[{"url":"https:\/\/a.nel.cloudflare.com\/report\/v4?s=3ELJyc7xSu8u4MGBQOyA9keQ3RPqAD%2FbBgtoUrGlhMLaWpcU81jr3Pnh5tNny1usAiZmWrr%2BURHsk%2FTpSq4OE7SOtC7k6qI5LJ79DWQk62nZ3VyfNJX4vosOULkwwMsg%2F%2BOOdQ%3D%3D"}],"group":"cf-nel","max_age":604800}
nel: {"success_fraction":0,"report_to":"cf-nel","max_age":604800}
vary: Accept-Encoding
server: cloudflare
cf-ray: 8bda003c799f2a94-LAX
content-encoding: br
5

I’ve fixed the brotli configuration issue (it was set in the wrong apache section), and while the above keycdn.com check reports it’s enabled, the content-encoding header still isn’t set.

This site shows it’s set:

But curl doesn’t show the content-encoding header:

$ curl -s -I -H 'Accept-Encoding: br' https://linuxsecurity.com/
HTTP/2 200 
date: Wed, 04 Sep 2024 02:29:51 GMT
content-type: text/html; charset=utf-8
expires: Wed, 17 Aug 2005 00:00:00 GMT
cache-control: max-age=120, no-transform
pragma: no-cache
vary: Accept-Encoding,User-Agent,Origin
x-content-encoded-by: Joomla
access-control-allow-origin: *
access-control-allow-methods: POST, GET, OPTIONS, DELETE, PUT
access-control-allow-headers: x-requested-with, Content-Type, origin, authorization, accept, client-security-token
access-control-expose-headers: Content-Security-Policy, Location
access-control-max-age: 600
strict-transport-security: max-age=31536000; includeSubDomains; preload
permissions-policy: accelerometer=(), autoplay=(), camera=(), cross-origin-isolated=(), display-capture=(), encrypted-media=(), fullscreen=(), geolocation=(), gyroscope=(), keyboard-map=(), magnetom
eter=(), microphone=(), midi=(), payment=(), picture-in-picture=(), publickey-credentials-get=(), screen-wake-lock=(), sync-xhr=(), usb=(), xr-spatial-tracking=(), clipboard-read=(), clipboard-write
=(), gamepad=(), hid=(), idle-detection=(), serial=(), window-placement=()
last-modified: Wed, 04 Sep 2024 02:28:56 GMT
etag: W/"57d6466c3a32238efb52b396d96e2bbe"
accept-encoding: br
cf-cache-status: HIT
age: 22
report-to: {"endpoints":[{"url":"https:\/\/a.nel.cloudflare.com\/report\/v4?s=J3HoVAOblqvpQji5XIzHIEERa6n9%2BBX%2Brbg5vqu62upbxp57%2FY9%2F2nhygBUVndVGOSBaA5VuNJmRct2OvxRdMmz8bQW7ffWFq3HmEMaNcBpVmAso
aN9Ox0nMd%2Fz2ioV6%2FwuJ"}],"group":"cf-nel","max_age":604800}
nel: {"success_fraction":0,"report_to":"cf-nel","max_age":604800}
expect-ct: max-age=86400, enforce
referrer-policy: same-origin
x-content-type-options: nosniff
x-frame-options: SAMEORIGIN
x-xss-protection: 1; mode=block
server: cloudflare
cf-ray: 8bda8de3ddd70cb4-EWR
7

So it turns out that this cache-control header was disabling brotli compression with cloudflare

       Header merge Cache-Control "no-transform"

I understand from this StackExchange post that this header effectively disables compression.

My concern is what the benefit (security or otherwise) I may be losing by disabling this header? I thought its intended purpose was to prevent intermediaries from changing cached web page resources. Is this a concern using cloudflare in 2024?

8

The Cache-Control: no-transform header is indeed used to prevent intermediaries (such as proxy servers or CDNs) from altering the content of cached resources. This could include transformations like compressing, resizing images, or modifying content in other ways. The header is primarily intended to ensure the integrity of resources as they pass through various points in the delivery chain.

However, in the context of using Cloudflare in 2024, especially with modern web technologies and better infrastructure management, the no-transform directive can sometimes conflict with optimizations like Brotli compression, as you’ve experienced. By disabling no-transform, you allow Cloudflare to apply its own performance optimizations, including Brotli compression, which can significantly improve load times for your users.

Potential Benefits of Disabling no-transform:

  1. Improved Performance: Brotli compression and other transformations by Cloudflare can reduce the size of resources, resulting in faster load times.
  2. Resource Optimization: Cloudflare may apply other optimizations (e.g., image compression or minification) that can further enhance the performance of your site.
  3. Smarter Caching: Allowing intermediaries like Cloudflare to modify resources ensures that they can make adjustments based on device type or network conditions, improving user experience.

Potential Concerns of Disabling no-transform:

  1. Integrity Concerns: If there is sensitive data or content that you do not want intermediaries to alter or compress (e.g., specific scripts or resources that need to remain untouched), removing no-transform could allow those changes.
  2. Edge Case Conflicts: In rare cases, certain transformations might lead to unintended behavior on the client-side, especially if your resources rely on a very specific format.

With Cloudflare, their transformations are generally reliable, and since they serve millions of websites, the risk of adverse effects is quite low. If your website doesn’t have strict content integrity requirements that mandate no transformations, allowing Cloudflare to apply optimizations like Brotli is usually beneficial for performance.

9

This topic was automatically closed 2 days after the last reply. New replies are no longer allowed.