No content-length header when Content-type: gzip

I am using Worker with FixedLengthStream as a proxy.
If the request to Worker contains the Accept-Encoding: gzip header, then the content-length is omitted in the response.

I’ve noticed similar issue here: No content-length header when response is compressed
I’ve tried do set: Cache-Control: no-transform header in the request to proxy, but content-length isn’t returned.

How can I force Cloudflare to return the content-length which shall match exactly value set to: FixedLengthStream?

Request without Accept-Encoding (works fine):

curl -v "https://test-proxy.syncawarestatic.workers.dev/?apiurl=https://s3.eu-central-003.backblazeb2.com/2uueviqfo2ci8tmifgbbvx/Screenshot%20from%202023-04-02%2018-38-30.png?X-Amz-Algorithm=AWS4-HMAC-SHA256%26X-Amz-Credential=00379f3243c252a0000000012%2F20230404%2Feu-central-003%2Fs3%2Faws4_request%26X-Amz-Date=20230404T105059Z%26X-Amz-Expires=604800%26X-Amz-SignedHeaders=host%26X-Amz-Signature=02d4785df8c255fe9c587ef2674a1d25ca63ca9e9abfe059b1432185a590841a"

< content-type: application/json;charset=utf-8
< content-length: 19448

Request with Accept-Encoding: gzip (missing the content-length):

curl -v "https://test-proxy.syncawarestatic.workers.dev/?apiurl=https://s3.eu-central-003.backblazeb2.com/2uueviqfo2ci8tmifgbbvx/Screenshot%20from%202023-04-02%2018-38-30.png?X-Amz-Algorithm=AWS4-HMAC-SHA256%26X-Amz-Credential=00379f3243c252a0000000012%2F20230404%2Feu-central-003%2Fs3%2Faws4_request%26X-Amz-Date=20230404T105059Z%26X-Amz-Expires=604800%26X-Amz-SignedHeaders=host%26X-Amz-Signature=02d4785df8c255fe9c587ef2674a1d25ca63ca9e9abfe059b1432185a590841a" -H 'Accept-Encoding: gzip'

< content-type: application/json;charset=utf-8

My worker:

export default {
    async fetch(req, env) {
        const requestUrl = new URL(req.url);
        const apiurl = requestUrl.searchParams.get('apiurl');
        let url = new URL(apiurl);
        let hostname = url.hostname;

        let headers = {
            host: hostname,
            origin: hostname
        }

        const response = await fetch(url, {
             headers: headers,
             method: req.method
        });

        let responseSize = response.headers.get("content-length");
        const { writable, readable } = new FixedLengthStream(responseSize);
        response.body.pipeTo(writable);
        return new Response(readable, response);
    }
}

Cloudflare performs transparent (de)compression based on your Accept-Encoding header - if a response has to be (de)compressed on-the-fly then the Content-Length cannot be known up-front and will be omitted.

2 Likes

Thanks for your response.
Given that my service worker is just a proxy acting as a middleman between sender and destination, and I have no control of the sender request and destination response is there any way to configure this behavior?
I rely on Content-Length header, as the response is streamed. I am fine with disabling compression altogether if that makes the Content-Length predictable.