CSV compression

Hello Cloudflare community,
We serve .csv files from our S3 bucket, through the Cloudflare CDN.
We would like to compress these files, and serve them.
Based on this article, we understand that the .csv files are not auto-compressed.
When we manually compress the .csv files,as .csv.gz, we do not see Cloudflare picking up the .gz files, when the .csv is requested for. Is there a configuration we need to perform, to make this happen?

https://support.cloudflare.com/hc/en-us/articles/200168396-What-will-Cloudflare-compress-

That is not how HTTP compression works.

If you make the following request to your origin, what response do you get? (replace the URL and Origin IP with your own details)

curl https://www.example.com/yourfile.csv --connect-to ::YOUR-ORIGIN-IP -H "Accept-Encoding: gzip" --dump-header - -o /dev/null --silent | egrep -i 'encoding|type'

2 Likes

My guess is that Cloudflare won’t compress the CSV filetype for the same reason that it doesn’t compress by default JPG, PNG, and other image, audio, and video file types — unless you pay for a subscription service such as Polish, Image Resize, or Stream or are on a paid plan that offers this feature. But while you can pay to have Polish or Stream, Cloudflare doesn’t have (so far) a paid service for CSV files or other downloadables.

Use of the Services for serving video or a disproportionate percentage of pictures, audio files, or other non-HTML content is prohibited, unless purchased separately as part of a Paid Service or expressly allowed under our Supplemental Terms for a specific Service. If we determine you have breached this Section 2.8, we may immediately suspend or restrict your use of the Services, or limit End User access to certain of your resources through the Services.

Keeping in mind the limitations quoted above, if you believe your use case falls within the accepted use as defined by the Cloudflare TOS for your plan level, what you can do as a workaround (if it doesn’t interfere with further processing of the file) is changing the Content-Type header value to text/richtext, which will make Cloudflare compress it, and set a header with “Content-Disposition: attachment” so that the file is actually downloaded, as opposed to displayed on the browser screen.

You can set both headers with Transform Rules.
https://dash.cloudflare.com/?to=/:account/:zone/rules/transform-rules

Create Transform Rule > Modify Response Header >

If URI-Path contains “.csv”

Add Static Headers:
Content-Type: text/richtext
Content-Disposition: attachment

On a test website, this has worked, so I hope it works for you as well.

1 Like

The reason they don’t compress media types is that they are essentially compressed already, and Gzip is a waste of resources for them. That is somewhat different to image compression, which is removing data from the files, or doing a format conversation.

In my tests, if the Origin sends a compressed version of the file, then Cloudflare will sent the compressed version to supporting clients, and decompress to other clients. If the Origin did a cache fill without gzip, then it was always uncompressed.

 % curl https://www.example.com/test.csv -H "Accept-Encoding: gzip"
HTTP/2 200
content-type: text/css
content-length: 464
vary: Accept-Encoding
content-encoding: gzip

% curl https://www.example.com/test.csv -H "Accept-Encoding: brotli"
HTTP/2 200
content-type: text/css
content-length: 8880
cf-bgj: minify
cf-polished: status=cannot_optimize
vary: Accept-Encoding

% curl https://www.example.com/test.csv -H "Accept-Encoding: identity"
HTTP/2 200
content-type: text/css
content-length: 8880
cf-bgj: minify
cf-polished: status=cannot_optimize
vary: Accept-Encoding

Not sure why CF would not just compress everything with text/*

2 Likes

Thank you, I set up this transform rule. Works as expected. The csv is compressed using brotli, when transferred on the wire.
Went with Content-Type: text/plain instead of richtext.

1 Like

I agree. Hope CF rolls this feature out.

1 Like

This topic was automatically closed 3 days after the last reply. New replies are no longer allowed.