Serving compressed content from workers

Hello,

In an application I’m building using CF Workers, I am retrieving a JSON (application/json) file from an origin, modifying it, and then serving it to the user. I am also, however, modifying the Content-Type header to conform to a spec, resulting in it being application/webpub+json. When I serve this JSON with that Content-Type, it is no longer served with any sort of compression, even if I manually add a Content-Encoding: gzip header. Does this have to do with the limited set of mimetypes listed on this page: https://support.cloudflare.com/hc/en-us/articles/200168396 ? And if so is it possible in any way to compress the application/webpub+json, or do I need to give up and just use application/json?

1 Like

I forgot, some code to demonstrate what I am doing:

        // Get the JSON body
        const manifest = await response.json();

        // Perform manifest modifications....

        const headers = utils.carryOverHeaders(response.headers);
        headers.set("Content-Type", `application/${action}+json; charset=UTF-8`);

        return new Response(JSON.stringify(manifest), { // But this response is not being compressed
            headers
        });

I simulated some application/webpub+json and application/json:

application/webpub+json, no compression.
application/json very small output, no compression, but there is no point compressing what already is small.

Maybe the Workers team will work on this. :+1:

Something that was confusing to me at first is that when you use the workers testing environment with wrangler preview --watch, the gzip compression content-encoding does work on those servers.

The main reason I’m interested in compressing is because there’s a lot of repetitive data in the JSON. Without compression (for example), I have ~9KB, with it’s ~1.5KB, which is great savings, for data similar to this:

{
    "@context": "http://readium.org/webpub/epub.jsonld",
    "metadata": {
        "@type": "ComicIssue",
        "identifier": "urn:uuid:3703aa1e-3ab0-4450-92c6-5fda3089a893",
        "id": 90,
        "issueNumber": 0,
        "title": "Chapter 1-1",
        "subtitle": "Chapter 1-1",
        "author": [{
            "@type": "Person",
            "name": "FUNATO Akari",
            "identifier": "1",
            "url": "[omitted]",
            "alternateName": "さとあきら"
        }],
        "artist": [{
            "@type": "Person",
            "name": "FUNATO Akari",
            "identifier": "1",
            "url": "[omitted]",
            "alternateName": "さとあきら"
        }],
        "accessMode": "visual",
        "accessibilityControl": ["fullKeyboardControl", "fullMouseControl", "fullTouchControl"],
        "isAccessibleForFree": true,
        "protection": null,
        "accessibilitySummary": "Sequence of images containing drawings with text",
        "provider": "[omitted]",
        "publisher": [{
            "name": "[omitted]",
            "identifier": "[omitted]"
        }],
        "description": "This is some pretty amazing dummy text",
        "published": "2018-07-05T19:08:57+00:00",
        "modified": "2018-07-05T19:12:13+00:00",
        "language": "en",
        "subject": ["Drama", "Historical", "Mystery", "Romance", "Seinen", "Slice of Life", "Tragedy"],
        "belongsTo": {
            "series": {
                "name": "Neon Genesis Evangelion",
                "slug": "nge",
                "identifier": "[omitted]",
                "id": 1,
                "position": "0"
            }
        },
        "numberOfPages": 32,
        "readingProgression": "rtl",
        "rendition": {
            "layout": "pre-paginated",
            "orientation": "portrait",
            "spread": "landscape"
        },
        "image": "https://[omitted]/static/media/7d071f55-b084-435b-8ecd-68b3806009f5/22924.png",
        "thumbnailUrl": "https://[omitted]/static/media/7d071f55-b084-435b-8ecd-68b3806009f5/22924.png?fit=250%2C250"
    },
    "links": [{
        "rel": "self",
        "href": "https://[omitted]/proj/r/static/5fc125d8-9eed-422b-a545-7716c01edb8f/manifest.json",
        "type": "application/webpub+json"
    }, {
        "rel": "alternate",
        "href": "https://[omitted]/proj/r/static/5fc125d8-9eed-422b-a545-7716c01edb8f/0/",
        "type": "text/html"
    }],
    "readingOrder": [{
        "href": "_p-cover.jpg?token=38b6bfcb327dbad4097ab844b235e331b74e80a105f0e627d2c7e2b06a2222b9&expires=1567498520&scope=%2Fdelivery%2F3703aa1e-3ab0-4450-92c6-5fda3089a893%2F",
        "type": "image/jpeg",
        "height": 2048,
        "width": 1445
    }, {
        "href": "p-001.jpg?token=38b6bfcb327dbad4097ab844b235e331b74e80a105f0e627d2c7e2b06a2222b9&expires=1567498520&scope=%2Fdelivery%2F3703aa1e-3ab0-4450-92c6-5fda3089a893%2F",
        "type": "image/jpeg",
        "height": 2048,
        "width": 1445
    }, {
        "href": "p-002.jpg?token=38b6bfcb327dbad4097ab844b235e331b74e80a105f0e627d2c7e2b06a2222b9&expires=1567498520&scope=%2Fdelivery%2F3703aa1e-3ab0-4450-92c6-5fda3089a893%2F",
        "type": "image/jpeg",
        "height": 2048,
        "width": 1445
    }, {
        "href": "p-003.jpg?token=38b6bfcb327dbad4097ab844b235e331b74e80a105f0e627d2c7e2b06a2222b9&expires=1567498520&scope=%2Fdelivery%2F3703aa1e-3ab0-4450-92c6-5fda3089a893%2F",
        "type": "image/jpeg",
        "height": 2048,
        "width": 1445
    }, {
        "href": "p-004.jpg?token=38b6bfcb327dbad4097ab844b235e331b74e80a105f0e627d2c7e2b06a2222b9&expires=1567498520&scope=%2Fdelivery%2F3703aa1e-3ab0-4450-92c6-5fda3089a893%2F",
        "type": "image/jpeg",
        "height": 2048,
        "width": 1445
    }, {
        "href": "p-005.jpg?token=38b6bfcb327dbad4097ab844b235e331b74e80a105f0e627d2c7e2b06a2222b9&expires=1567498520&scope=%2Fdelivery%2F3703aa1e-3ab0-4450-92c6-5fda3089a893%2F",
        "type": "image/jpeg",
        "height": 2048,
        "width": 1445
    }, {
        "href": "p-006.jpg?token=38b6bfcb327dbad4097ab844b235e331b74e80a105f0e627d2c7e2b06a2222b9&expires=1567498520&scope=%2Fdelivery%2F3703aa1e-3ab0-4450-92c6-5fda3089a893%2F",
        "type": "image/jpeg",
        "height": 2048,
        "width": 1445
    }, {
        "href": "p-007.jpg?token=38b6bfcb327dbad4097ab844b235e331b74e80a105f0e627d2c7e2b06a2222b9&expires=1567498520&scope=%2Fdelivery%2F3703aa1e-3ab0-4450-92c6-5fda3089a893%2F",
        "type": "image/jpeg",
        "height": 2048,
        "width": 1445
    }, {
        "href": "p-008.jpg?token=38b6bfcb327dbad4097ab844b235e331b74e80a105f0e627d2c7e2b06a2222b9&expires=1567498520&scope=%2Fdelivery%2F3703aa1e-3ab0-4450-92c6-5fda3089a893%2F",
        "type": "image/jpeg",
        "height": 2048,
        "width": 1445
    }, {
        "href": "p-009.jpg?token=38b6bfcb327dbad4097ab844b235e331b74e80a105f0e627d2c7e2b06a2222b9&expires=1567498520&scope=%2Fdelivery%2F3703aa1e-3ab0-4450-92c6-5fda3089a893%2F",
        "type": "image/jpeg",
        "height": 2048,
        "width": 1445
    }, {
        "href": "p-010.jpg?token=38b6bfcb327dbad4097ab844b235e331b74e80a105f0e627d2c7e2b06a2222b9&expires=1567498520&scope=%2Fdelivery%2F3703aa1e-3ab0-4450-92c6-5fda3089a893%2F",
        "type": "image/jpeg",
        "height": 2048,
        "width": 1445
    }, {
        "href": "p-011.jpg?token=38b6bfcb327dbad4097ab844b235e331b74e80a105f0e627d2c7e2b06a2222b9&expires=1567498520&scope=%2Fdelivery%2F3703aa1e-3ab0-4450-92c6-5fda3089a893%2F",
        "type": "image/jpeg",
        "height": 2048,
        "width": 1445
    }, {
        "href": "p-012.jpg?token=38b6bfcb327dbad4097ab844b235e331b74e80a105f0e627d2c7e2b06a2222b9&expires=1567498520&scope=%2Fdelivery%2F3703aa1e-3ab0-4450-92c6-5fda3089a893%2F",
        "type": "image/jpeg",
        "height": 2048,
        "width": 1445
    }, {
        "href": "p-013.jpg?token=38b6bfcb327dbad4097ab844b235e331b74e80a105f0e627d2c7e2b06a2222b9&expires=1567498520&scope=%2Fdelivery%2F3703aa1e-3ab0-4450-92c6-5fda3089a893%2F",
        "type": "image/jpeg",
        "height": 2048,
        "width": 1445
    }, {
        "href": "p-014.jpg?token=38b6bfcb327dbad4097ab844b235e331b74e80a105f0e627d2c7e2b06a2222b9&expires=1567498520&scope=%2Fdelivery%2F3703aa1e-3ab0-4450-92c6-5fda3089a893%2F",
        "type": "image/jpeg",
        "height": 2048,
        "width": 1445
    }, {
        "href": "p-015.jpg?token=38b6bfcb327dbad4097ab844b235e331b74e80a105f0e627d2c7e2b06a2222b9&expires=1567498520&scope=%2Fdelivery%2F3703aa1e-3ab0-4450-92c6-5fda3089a893%2F",
        "type": "image/jpeg",
        "height": 2048,
        "width": 1445
    }, {
        "href": "p-016.jpg?token=38b6bfcb327dbad4097ab844b235e331b74e80a105f0e627d2c7e2b06a2222b9&expires=1567498520&scope=%2Fdelivery%2F3703aa1e-3ab0-4450-92c6-5fda3089a893%2F",
        "type": "image/jpeg",
        "height": 2048,
        "width": 1445
    }, {
        "href": "p-017.jpg?token=38b6bfcb327dbad4097ab844b235e331b74e80a105f0e627d2c7e2b06a2222b9&expires=1567498520&scope=%2Fdelivery%2F3703aa1e-3ab0-4450-92c6-5fda3089a893%2F",
        "type": "image/jpeg",
        "height": 2048,
        "width": 1445
    }, {
        "href": "p-018.jpg?token=38b6bfcb327dbad4097ab844b235e331b74e80a105f0e627d2c7e2b06a2222b9&expires=1567498520&scope=%2Fdelivery%2F3703aa1e-3ab0-4450-92c6-5fda3089a893%2F",
        "type": "image/jpeg",
        "height": 2048,
        "width": 1445
    }, {
        "href": "p-019.jpg?token=38b6bfcb327dbad4097ab844b235e331b74e80a105f0e627d2c7e2b06a2222b9&expires=1567498520&scope=%2Fdelivery%2F3703aa1e-3ab0-4450-92c6-5fda3089a893%2F",
        "type": "image/jpeg",
        "height": 2048,
        "width": 1445
    }, {
        "href": "p-020.jpg?token=38b6bfcb327dbad4097ab844b235e331b74e80a105f0e627d2c7e2b06a2222b9&expires=1567498520&scope=%2Fdelivery%2F3703aa1e-3ab0-4450-92c6-5fda3089a893%2F",
        "type": "image/jpeg",
        "height": 2048,
        "width": 1445
    }, {
        "href": "p-021.jpg?token=38b6bfcb327dbad4097ab844b235e331b74e80a105f0e627d2c7e2b06a2222b9&expires=1567498520&scope=%2Fdelivery%2F3703aa1e-3ab0-4450-92c6-5fda3089a893%2F",
        "type": "image/jpeg",
        "height": 2048,
        "width": 1445
    }, {
        "href": "p-022.jpg?token=38b6bfcb327dbad4097ab844b235e331b74e80a105f0e627d2c7e2b06a2222b9&expires=1567498520&scope=%2Fdelivery%2F3703aa1e-3ab0-4450-92c6-5fda3089a893%2F",
        "type": "image/jpeg",
        "height": 2048,
        "width": 1445
    }, {
        "href": "p-023.jpg?token=38b6bfcb327dbad4097ab844b235e331b74e80a105f0e627d2c7e2b06a2222b9&expires=1567498520&scope=%2Fdelivery%2F3703aa1e-3ab0-4450-92c6-5fda3089a893%2F",
        "type": "image/jpeg",
        "height": 2048,
        "width": 1445
    }, {
        "href": "p-024.jpg?token=38b6bfcb327dbad4097ab844b235e331b74e80a105f0e627d2c7e2b06a2222b9&expires=1567498520&scope=%2Fdelivery%2F3703aa1e-3ab0-4450-92c6-5fda3089a893%2F",
        "type": "image/jpeg",
        "height": 2048,
        "width": 1445
    }, {
        "href": "p-025.jpg?token=38b6bfcb327dbad4097ab844b235e331b74e80a105f0e627d2c7e2b06a2222b9&expires=1567498520&scope=%2Fdelivery%2F3703aa1e-3ab0-4450-92c6-5fda3089a893%2F",
        "type": "image/jpeg",
        "height": 2048,
        "width": 1445
    }, {
        "href": "p-026.jpg?token=38b6bfcb327dbad4097ab844b235e331b74e80a105f0e627d2c7e2b06a2222b9&expires=1567498520&scope=%2Fdelivery%2F3703aa1e-3ab0-4450-92c6-5fda3089a893%2F",
        "type": "image/jpeg",
        "height": 2048,
        "width": 1445
    }, {
        "href": "p-027.jpg?token=38b6bfcb327dbad4097ab844b235e331b74e80a105f0e627d2c7e2b06a2222b9&expires=1567498520&scope=%2Fdelivery%2F3703aa1e-3ab0-4450-92c6-5fda3089a893%2F",
        "type": "image/jpeg",
        "height": 2048,
        "width": 1445
    }, {
        "href": "p-028.jpg?token=38b6bfcb327dbad4097ab844b235e331b74e80a105f0e627d2c7e2b06a2222b9&expires=1567498520&scope=%2Fdelivery%2F3703aa1e-3ab0-4450-92c6-5fda3089a893%2F",
        "type": "image/jpeg",
        "height": 2048,
        "width": 1445
    }, {
        "href": "p-029.jpg?token=38b6bfcb327dbad4097ab844b235e331b74e80a105f0e627d2c7e2b06a2222b9&expires=1567498520&scope=%2Fdelivery%2F3703aa1e-3ab0-4450-92c6-5fda3089a893%2F",
        "type": "image/jpeg",
        "height": 2048,
        "width": 1445
    }, {
        "href": "p-030.jpg?token=38b6bfcb327dbad4097ab844b235e331b74e80a105f0e627d2c7e2b06a2222b9&expires=1567498520&scope=%2Fdelivery%2F3703aa1e-3ab0-4450-92c6-5fda3089a893%2F",
        "type": "image/jpeg",
        "height": 2048,
        "width": 1445
    }, {
        "href": "https://[omitted]/manga-lp/a687eaab-7a98-4448-a769-b97670a9821b.html?token=38b6bfcb327dbad4097ab844b235e331b74e80a105f0e627d2c7e2b06a2222b9&expires=1567498520&scope=%2Fdelivery%2F3703aa1e-3ab0-4450-92c6-5fda3089a893%2F",
        "type": "text/html",
        "properties": {
            "orientation": "landscape",
            "page": "center",
            "spread": "both",
            "layout": "fixed"
        },
        "title": "Last page",
        "xbr": {
            "final": true
        }
    }]
}

Is there a list somewhere of the headers that will be compressed? I was able to find this for Brotli compression in Cloudflare’s general product but, given that CF Workers don’t support Brotli, the compression semantics for CF Workers appear to be undocumented.

This topic was automatically closed after 14 days. New replies are no longer allowed.