How to decompress original server content to CF Edge cache

pzika · January 7, 2021, 6:26pm

I have gzipped assets at my Origin and have the following headers:

Cache-Control: public, max-age = 1800
Content-Type: application/octet-stream
Content-Encoding: gzip

Official Cloudflare documentation at

declares:

Cloudflare’s reverse proxy is also able to convert between compressed
formats and uncompressed formats, meaning that it can ** pull content
from a customer’s origin server via gzip and serve it to clients
uncompressed ** (or vice versa). This is done independently of
caching.

but when Cloudflare caches my assets, they remain gzipped. My web application use Range Request so I need uncompressed assets in Edge cache.

Question

Is it possible to have compressed asset at original server and uncompressed asset at Cloudflare CDN cache?
What headers x Cloudflare settings are needed?

pzika · January 7, 2021, 7:01pm

I have similar problem, but without worker:

What headers x cloudflare settings are needed to obtain decompress data at CDN cache when the original server sends compressed data with gzip / br / deflate content encoding?

My https://origin-server/entries.bin data with following headers remains compressed at CDN cache:

Cache-Control: public, max-age=630720000
Content-Type: application/octet-stream
Content-Encoding: gzip

eva2000 · January 7, 2021, 7:06pm

All you need for uncompressed asset served from CF edge server is to request the asset without the Accept-Encoding request HTTP header

with Accept-Encoding request HTTP header

curl -I -H "accept-encoding: gzip, deflate, br" https://www.cloudflare.com/
HTTP/2 200 
date: Thu, 07 Jan 2021 19:02:43 GMT
content-type: text/html; charset=utf-8
set-cookie: __cfduid=d4e9d389f06c4f4a127cf36d2afd4b1e61610046163; expires=Sat, 06-Feb-21 19:02:43 GMT; path=/; domain=.www.cloudflare.com; HttpOnly; SameSite=Lax; Secure
cf-ray: 60dfef494bea4bd0-YUL
age: 47
cache-control: public, max-age=30
strict-transport-security: max-age=31536000
vary: Accept-Encoding
cf-cache-status: HIT
cf-request-id: 077fd3e1cf00004bd037131000000001
expect-ct: max-age=604800, report-uri="https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct"
x-content-type-options: nosniff
x-frame-options: SAMEORIGIN
x-xss-protection: 1; mode=block
report-to: {"endpoints":[{"url":"https:\/\/a.nel.cloudflare.com\/report?s=iDr5cBJe0rnJ2mZr%2B0GMxt0jdyVOVDrkzuEeZaxK9B0QScKH%2BPU6drfJt0W%2BfNXeuDkszfNyDOxHSmyhFWnjKC6%2FKmJfI4GZzuCSGMeGK4iIlASGWC%2BY5Z%2BmM0PE3Cc%3D"}],"group":"cf-nel","max_age":604800}
nel: {"report_to":"cf-nel","max_age":604800}
server: cloudflare
content-encoding: br

without Accept-Encoding request HTTP header

curl -I https://www.cloudflare.com/                                        
HTTP/2 200 
date: Thu, 07 Jan 2021 19:03:53 GMT
content-type: text/html; charset=utf-8
set-cookie: __cfduid=d5cbe2a4b3a6c9e5662438a01fc7c1bbd1610046233; expires=Sat, 06-Feb-21 19:03:53 GMT; path=/; domain=.www.cloudflare.com; HttpOnly; SameSite=Lax; Secure
cf-ray: 60dff0fe2e944bd1-YUL
age: 56
cache-control: public, max-age=30
strict-transport-security: max-age=31536000
vary: Accept-Encoding
cf-cache-status: HIT
cf-request-id: 077fd4f2d900004bd1da944000000001
expect-ct: max-age=604800, report-uri="https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct"
x-content-type-options: nosniff
x-frame-options: SAMEORIGIN
x-xss-protection: 1; mode=block
report-to: {"endpoints":[{"url":"https:\/\/a.nel.cloudflare.com\/report?s=ESH0JA%2FfyZJMDumS4qDMYobxuCQlH4qNZQWg4F%2FpEZ9TClxv8H0UO%2Ffk%2BJ6hWyFL4%2FOA2iiySHiiEsiKQUbu6nq%2BzEHd6bihRm5tCZh9F2yO%2FtAaZzSoeaHMROSOgyk%3D"}],"group":"cf-nel","max_age":604800}
nel: {"report_to":"cf-nel","max_age":604800}
server: cloudflare

the verbose response with Accept-Encoding request HTTP header

curl -Iv -H "accept-encoding: gzip, deflate, br" https://www.cloudflare.com/
*   Trying 2606:4700::6810:7c60:443...
* Connected to www.cloudflare.com (2606:4700::6810:7c60) port 443 (#0)
* Initializing NSS with certpath: sql:/etc/pki/nssdb
*  CAfile: none
*  CApath: none
* loaded libnssckbi.so
* ALPN, server accepted to use h2
* SSL connection using TLS_AES_128_GCM_SHA256
* Server certificate:
*       subject: CN=www.cloudflare.com,O="Cloudflare, Inc.",L=San Francisco,ST=CA,C=US
*       start date: Oct 19 00:00:00 2020 GMT
*       expire date: Oct 18 23:59:59 2021 GMT
*       common name: www.cloudflare.com
*       issuer: CN=Cloudflare Inc ECC CA-3,O="Cloudflare, Inc.",C=US
* Using HTTP2, server supports multi-use
* Connection state changed (HTTP/2 confirmed)
* Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0
* Using Stream ID: 1 (easy handle 0x1478810)
> HEAD / HTTP/2
> Host: www.cloudflare.com
> user-agent: curl/7.73.0
> accept: */*
> accept-encoding: gzip, deflate, br
> 
* Connection state changed (MAX_CONCURRENT_STREAMS == 256)!
< HTTP/2 200 
HTTP/2 200

pzika · January 8, 2021, 1:37pm

I am sorry, but I do not understand. My web application use Range Request so I need uncompressed assets at CF CDN (at Edge server).

I have:

gzipped content (Content-Encoding: gzip) at original server
a web page with the following request against CF CDN:

     var xhr = new XMLHttpRequest ();
     xhr.open ('GET', url, true);
     xhr.setRequestHeader ('Range', `bytes = 1000-2000`);
     xhr.setRequestHeader ('Cache-Control', 'no-cache');
     xhr2.onreadystatechange = function () {};
     xhr.send ();

Problem

Because the CF CDN contains gzipped content, XMLHttpRequest returns the meaningless range from gzip and not the desired range from unzipped data.

I don’t know if it’s important, but assets have a .proto extension.

pzika · January 8, 2021, 1:35pm

More infos

1. Chrome adds “accept-encoding: identity”

… for Ajax Range request, Chrome browser adds accept-encoding: identity to request header

2. CF CDN url for testing: https://data.wikibulary.com/$web/en.proto

Length=4946405, GZipped length = 2108616
When downloaded from Chrome browser, correct 4946405-length file saved to local disk.

Origin server headers:

Cache-Control: public, max-age = 630720000
Content-Type: application/x-protobuf
Content-Encoding: gzip

3. pure ajax GET request

Status code 200

Some of response headers:

accept-ranges: bytes
cf-cache-status: HIT
cache-control: public, max-age=630720000
content-encoding: gzip
content-length: 2108616
content-type: application/x-protobuf

4. Ajax range request with “Range: bytes=4946404-4946404” header

… trying to return last byte of uncompressed version
Status code 416 ( Range Not Satisfiable)

Some of response headers:

content-type: text/html
content-range: bytes */2108616

5. Ajax range request with “Range: bytes=2108615-2108615” header

… returns last byte of compressed version
Status code 206

Some of response headers:

content-type: application/x-protobuf
Content-Range: bytes 2108615-2108615/2108616

system · January 13, 2021, 8:54am

This topic was automatically closed 5 days after the last reply. New replies are no longer allowed.