Serving compressed content from workers

Developers Workers
1

Hello,

In an application I’m building using CF Workers, I am retrieving a JSON (application/json) file from an origin, modifying it, and then serving it to the user. I am also, however, modifying the Content-Type header to conform to a spec, resulting in it being application/webpub+json. When I serve this JSON with that Content-Type, it is no longer served with any sort of compression, even if I manually add a Content-Encoding: gzip header. Does this have to do with the limited set of mimetypes listed on this page: https://support.cloudflare.com/hc/en-us/articles/200168396 ? And if so is it possible in any way to compress the application/webpub+json, or do I need to give up and just use application/json?

1 Like
2

I forgot, some code to demonstrate what I am doing:

        // Get the JSON body
        const manifest = await response.json();

        // Perform manifest modifications....

        const headers = utils.carryOverHeaders(response.headers);
        headers.set("Content-Type", `application/${action}+json; charset=UTF-8`);

        return new Response(JSON.stringify(manifest), { // But this response is not being compressed
            headers
        });
3

I simulated some application/webpub+json and application/json:

application/webpub+json, no compression.
application/json very small output, no compression, but there is no point compressing what already is small.

Maybe the Workers team will work on this. :+1:

4

Something that was confusing to me at first is that when you use the workers testing environment with wrangler preview --watch, the gzip compression content-encoding does work on those servers.

The main reason I’m interested in compressing is because there’s a lot of repetitive data in the JSON. Without compression (for example), I have ~9KB, with it’s ~1.5KB, which is great savings, for data similar to this:

{
    "@context": "http://readium.org/webpub/epub.jsonld",
    "metadata": {
        "@type": "ComicIssue",
        "identifier": "urn:uuid:3703aa1e-3ab0-4450-92c6-5fda3089a893",
        "id": 90,
        "issueNumber": 0,
        "title": "Chapter 1-1",
        "subtitle": "Chapter 1-1",
        "author": [{
            "@type": "Person",
            "name": "FUNATO Akari",
            "identifier": "1",
            "url": "[omitted]",
            "alternateName": "さとあきら"
        }],
        "artist": [{
            "@type": "Person",
            "name": "FUNATO Akari",
            "identifier": "1",
            "url": "[omitted]",
            "alternateName": "さとあきら"
        }],
        "accessMode": "visual",
        "accessibilityControl": ["fullKeyboardControl", "fullMouseControl", "fullTouchControl"],
        "isAccessibleForFree": true,
        "protection": null,
        "accessibilitySummary": "Sequence of images containing drawings with text",
        "provider": "[omitted]",
        "publisher": [{
            "name": "[omitted]",
            "identifier": "[omitted]"
        }],
        "description": "This is some pretty amazing dummy text",
        "published": "2018-07-05T19:08:57+00:00",
        "modified": "2018-07-05T19:12:13+00:00",
        "language": "en",
        "subject": ["Drama", "Historical", "Mystery", "Romance", "Seinen", "Slice of Life", "Tragedy"],
        "belongsTo": {
            "series": {
                "name": "Neon Genesis Evangelion",
                "slug": "nge",
                "identifier": "[omitted]",
                "id": 1,
                "position": "0"
            }
        },
        "numberOfPages": 32,
        "readingProgression": "rtl",
        "rendition": {
            "layout": "pre-paginated",
            "orientation": "portrait",
            "spread": "landscape"
        },
        "image": "https://[omitted]/static/media/7d071f55-b084-435b-8ecd-68b3806009f5/22924.png",
        "thumbnailUrl": "https://[omitted]/static/media/7d071f55-b084-435b-8ecd-68b3806009f5/22924.png?fit=250%2C250"
    },
    "links": [{
        "rel": "self",
        "href": "https://[omitted]/proj/r/static/5fc125d8-9eed-422b-a545-7716c01edb8f/manifest.json",
        "type": "application/webpub+json"
    }, {
        "rel": "alternate",
        "href": "https://[omitted]/proj/r/static/5fc125d8-9eed-422b-a545-7716c01edb8f/0/",
        "type": "text/html"
    }],
    "readingOrder": [{
        "href": "_p-cover.jpg?token=38b6bfcb327dbad4097ab844b235e331b74e80a105f0e627d2c7e2b06a2222b9&expires=1567498520&scope=%2Fdelivery%2F3703aa1e-3ab0-4450-92c6-5fda3089a893%2F",
        "type": "image/jpeg",
        "height": 2048,
        "width": 1445
    }, {
        "href": "p-001.jpg?token=38b6bfcb327dbad4097ab844b235e331b74e80a105f0e627d2c7e2b06a2222b9&expires=1567498520&scope=%2Fdelivery%2F3703aa1e-3ab0-4450-92c6-5fda3089a893%2F",
        "type": "image/jpeg",
        "height": 2048,
        "width": 1445
    }, {
        "href": "p-002.jpg?token=38b6bfcb327dbad4097ab844b235e331b74e80a105f0e627d2c7e2b06a2222b9&expires=1567498520&scope=%2Fdelivery%2F3703aa1e-3ab0-4450-92c6-5fda3089a893%2F",
        "type": "image/jpeg",
        "height": 2048,
        "width": 1445
    }, {
        "href": "p-003.jpg?token=38b6bfcb327dbad4097ab844b235e331b74e80a105f0e627d2c7e2b06a2222b9&expires=1567498520&scope=%2Fdelivery%2F3703aa1e-3ab0-4450-92c6-5fda3089a893%2F",
        "type": "image/jpeg",
        "height": 2048,
        "width": 1445
    }, {
        "href": "p-004.jpg?token=38b6bfcb327dbad4097ab844b235e331b74e80a105f0e627d2c7e2b06a2222b9&expires=1567498520&scope=%2Fdelivery%2F3703aa1e-3ab0-4450-92c6-5fda3089a893%2F",
        "type": "image/jpeg",
        "height": 2048,
        "width": 1445
    }, {
        "href": "p-005.jpg?token=38b6bfcb327dbad4097ab844b235e331b74e80a105f0e627d2c7e2b06a2222b9&expires=1567498520&scope=%2Fdelivery%2F3703aa1e-3ab0-4450-92c6-5fda3089a893%2F",
        "type": "image/jpeg",
        "height": 2048,
        "width": 1445
    }, {
        "href": "p-006.jpg?token=38b6bfcb327dbad4097ab844b235e331b74e80a105f0e627d2c7e2b06a2222b9&expires=1567498520&scope=%2Fdelivery%2F3703aa1e-3ab0-4450-92c6-5fda3089a893%2F",
        "type": "image/jpeg",
        "height": 2048,
        "width": 1445
    }, {
        "href": "p-007.jpg?token=38b6bfcb327dbad4097ab844b235e331b74e80a105f0e627d2c7e2b06a2222b9&expires=1567498520&scope=%2Fdelivery%2F3703aa1e-3ab0-4450-92c6-5fda3089a893%2F",
        "type": "image/jpeg",
        "height": 2048,
        "width": 1445
    }, {
        "href": "p-008.jpg?token=38b6bfcb327dbad4097ab844b235e331b74e80a105f0e627d2c7e2b06a2222b9&expires=1567498520&scope=%2Fdelivery%2F3703aa1e-3ab0-4450-92c6-5fda3089a893%2F",
        "type": "image/jpeg",
        "height": 2048,
        "width": 1445
    }, {
        "href": "p-009.jpg?token=38b6bfcb327dbad4097ab844b235e331b74e80a105f0e627d2c7e2b06a2222b9&expires=1567498520&scope=%2Fdelivery%2F3703aa1e-3ab0-4450-92c6-5fda3089a893%2F",
        "type": "image/jpeg",
        "height": 2048,
        "width": 1445
    }, {
        "href": "p-010.jpg?token=38b6bfcb327dbad4097ab844b235e331b74e80a105f0e627d2c7e2b06a2222b9&expires=1567498520&scope=%2Fdelivery%2F3703aa1e-3ab0-4450-92c6-5fda3089a893%2F",
        "type": "image/jpeg",
        "height": 2048,
        "width": 1445
    }, {
        "href": "p-011.jpg?token=38b6bfcb327dbad4097ab844b235e331b74e80a105f0e627d2c7e2b06a2222b9&expires=1567498520&scope=%2Fdelivery%2F3703aa1e-3ab0-4450-92c6-5fda3089a893%2F",
        "type": "image/jpeg",
        "height": 2048,
        "width": 1445
    }, {
        "href": "p-012.jpg?token=38b6bfcb327dbad4097ab844b235e331b74e80a105f0e627d2c7e2b06a2222b9&expires=1567498520&scope=%2Fdelivery%2F3703aa1e-3ab0-4450-92c6-5fda3089a893%2F",
        "type": "image/jpeg",
        "height": 2048,
        "width": 1445
    }, {
        "href": "p-013.jpg?token=38b6bfcb327dbad4097ab844b235e331b74e80a105f0e627d2c7e2b06a2222b9&expires=1567498520&scope=%2Fdelivery%2F3703aa1e-3ab0-4450-92c6-5fda3089a893%2F",
        "type": "image/jpeg",
        "height": 2048,
        "width": 1445
    }, {
        "href": "p-014.jpg?token=38b6bfcb327dbad4097ab844b235e331b74e80a105f0e627d2c7e2b06a2222b9&expires=1567498520&scope=%2Fdelivery%2F3703aa1e-3ab0-4450-92c6-5fda3089a893%2F",
        "type": "image/jpeg",
        "height": 2048,
        "width": 1445
    }, {
        "href": "p-015.jpg?token=38b6bfcb327dbad4097ab844b235e331b74e80a105f0e627d2c7e2b06a2222b9&expires=1567498520&scope=%2Fdelivery%2F3703aa1e-3ab0-4450-92c6-5fda3089a893%2F",
        "type": "image/jpeg",
        "height": 2048,
        "width": 1445
    }, {
        "href": "p-016.jpg?token=38b6bfcb327dbad4097ab844b235e331b74e80a105f0e627d2c7e2b06a2222b9&expires=1567498520&scope=%2Fdelivery%2F3703aa1e-3ab0-4450-92c6-5fda3089a893%2F",
        "type": "image/jpeg",
        "height": 2048,
        "width": 1445
    }, {
        "href": "p-017.jpg?token=38b6bfcb327dbad4097ab844b235e331b74e80a105f0e627d2c7e2b06a2222b9&expires=1567498520&scope=%2Fdelivery%2F3703aa1e-3ab0-4450-92c6-5fda3089a893%2F",
        "type": "image/jpeg",
        "height": 2048,
        "width": 1445
    }, {
        "href": "p-018.jpg?token=38b6bfcb327dbad4097ab844b235e331b74e80a105f0e627d2c7e2b06a2222b9&expires=1567498520&scope=%2Fdelivery%2F3703aa1e-3ab0-4450-92c6-5fda3089a893%2F",
        "type": "image/jpeg",
        "height": 2048,
        "width": 1445
    }, {
        "href": "p-019.jpg?token=38b6bfcb327dbad4097ab844b235e331b74e80a105f0e627d2c7e2b06a2222b9&expires=1567498520&scope=%2Fdelivery%2F3703aa1e-3ab0-4450-92c6-5fda3089a893%2F",
        "type": "image/jpeg",
        "height": 2048,
        "width": 1445
    }, {
        "href": "p-020.jpg?token=38b6bfcb327dbad4097ab844b235e331b74e80a105f0e627d2c7e2b06a2222b9&expires=1567498520&scope=%2Fdelivery%2F3703aa1e-3ab0-4450-92c6-5fda3089a893%2F",
        "type": "image/jpeg",
        "height": 2048,
        "width": 1445
    }, {
        "href": "p-021.jpg?token=38b6bfcb327dbad4097ab844b235e331b74e80a105f0e627d2c7e2b06a2222b9&expires=1567498520&scope=%2Fdelivery%2F3703aa1e-3ab0-4450-92c6-5fda3089a893%2F",
        "type": "image/jpeg",
        "height": 2048,
        "width": 1445
    }, {
        "href": "p-022.jpg?token=38b6bfcb327dbad4097ab844b235e331b74e80a105f0e627d2c7e2b06a2222b9&expires=1567498520&scope=%2Fdelivery%2F3703aa1e-3ab0-4450-92c6-5fda3089a893%2F",
        "type": "image/jpeg",
        "height": 2048,
        "width": 1445
    }, {
        "href": "p-023.jpg?token=38b6bfcb327dbad4097ab844b235e331b74e80a105f0e627d2c7e2b06a2222b9&expires=1567498520&scope=%2Fdelivery%2F3703aa1e-3ab0-4450-92c6-5fda3089a893%2F",
        "type": "image/jpeg",
        "height": 2048,
        "width": 1445
    }, {
        "href": "p-024.jpg?token=38b6bfcb327dbad4097ab844b235e331b74e80a105f0e627d2c7e2b06a2222b9&expires=1567498520&scope=%2Fdelivery%2F3703aa1e-3ab0-4450-92c6-5fda3089a893%2F",
        "type": "image/jpeg",
        "height": 2048,
        "width": 1445
    }, {
        "href": "p-025.jpg?token=38b6bfcb327dbad4097ab844b235e331b74e80a105f0e627d2c7e2b06a2222b9&expires=1567498520&scope=%2Fdelivery%2F3703aa1e-3ab0-4450-92c6-5fda3089a893%2F",
        "type": "image/jpeg",
        "height": 2048,
        "width": 1445
    }, {
        "href": "p-026.jpg?token=38b6bfcb327dbad4097ab844b235e331b74e80a105f0e627d2c7e2b06a2222b9&expires=1567498520&scope=%2Fdelivery%2F3703aa1e-3ab0-4450-92c6-5fda3089a893%2F",
        "type": "image/jpeg",
        "height": 2048,
        "width": 1445
    }, {
        "href": "p-027.jpg?token=38b6bfcb327dbad4097ab844b235e331b74e80a105f0e627d2c7e2b06a2222b9&expires=1567498520&scope=%2Fdelivery%2F3703aa1e-3ab0-4450-92c6-5fda3089a893%2F",
        "type": "image/jpeg",
        "height": 2048,
        "width": 1445
    }, {
        "href": "p-028.jpg?token=38b6bfcb327dbad4097ab844b235e331b74e80a105f0e627d2c7e2b06a2222b9&expires=1567498520&scope=%2Fdelivery%2F3703aa1e-3ab0-4450-92c6-5fda3089a893%2F",
        "type": "image/jpeg",
        "height": 2048,
        "width": 1445
    }, {
        "href": "p-029.jpg?token=38b6bfcb327dbad4097ab844b235e331b74e80a105f0e627d2c7e2b06a2222b9&expires=1567498520&scope=%2Fdelivery%2F3703aa1e-3ab0-4450-92c6-5fda3089a893%2F",
        "type": "image/jpeg",
        "height": 2048,
        "width": 1445
    }, {
        "href": "p-030.jpg?token=38b6bfcb327dbad4097ab844b235e331b74e80a105f0e627d2c7e2b06a2222b9&expires=1567498520&scope=%2Fdelivery%2F3703aa1e-3ab0-4450-92c6-5fda3089a893%2F",
        "type": "image/jpeg",
        "height": 2048,
        "width": 1445
    }, {
        "href": "https://[omitted]/manga-lp/a687eaab-7a98-4448-a769-b97670a9821b.html?token=38b6bfcb327dbad4097ab844b235e331b74e80a105f0e627d2c7e2b06a2222b9&expires=1567498520&scope=%2Fdelivery%2F3703aa1e-3ab0-4450-92c6-5fda3089a893%2F",
        "type": "text/html",
        "properties": {
            "orientation": "landscape",
            "page": "center",
            "spread": "both",
            "layout": "fixed"
        },
        "title": "Last page",
        "xbr": {
            "final": true
        }
    }]
}
5

Is there a list somewhere of the headers that will be compressed? I was able to find this for Brotli compression in Cloudflare’s general product but, given that CF Workers don’t support Brotli, the compression semantics for CF Workers appear to be undocumented.

1 Like
6

This topic was automatically closed after 14 days. New replies are no longer allowed.