What is the name of the domain?
*.rainforestqa.com (but very visible with vmanager.rainforestqa.com)
What is the error message?
Zlib::DataError incorrect header check (Zlib::DataError)
What is the issue you’re encountering
We have a number of ruby applications that make requests to sites hosted behind cloudflare. For the last week or two we’re periodically getting responses back which specify a Content-Encoding of gzip, but the response body is uncompressed. This is happening for a number of our origins, including an s3 bucket being proxied behind cloudflare, so I do not think this is a configuration error on our server side. This is happening across a number of our cloudflare domains.
What steps have you taken to resolve the issue?
We have tried to put a compression rule in place to force gzip (disabling zstd/brotli), but this made no change. We recently placed a compression rule disabling compression entirely to see if we could temporarily work around the issue, and that seems to be working for now, but I don’t think this is a long term solution.
We added additional logging in our ruby code to confirm the issue, and this is what we see for the response headers (note the content-encoding: gzip).
Headers: {
"date"=>["Sat, 18 Jan 2025 14:19:13 GMT"],
"content-type"=>["application/json; charset=utf-8"],
"transfer-encoding"=>["chunked"],
"connection"=>["close"],
"content-encoding"=>["gzip"],
"cf-ray"=>["903f38020be1afef-ATL"],
"x-frame-options"=>["SAMEORIGIN"],
"x-xss-protection"=>["0"],
"x-content-type-options"=>["nosniff"],
"x-permitted-cross-domain-policies"=>["none"],
"referrer-policy"=>["strict-origin-when-cross-origin"],
"vary"=>["Accept"],
"etag"=>["W/\"423f8a7c024bb31fb9017532d922b47a\""],
"cache-control"=>["max-age=0, private, must-revalidate"],
"x-request-id"=>["667c3dbd-cbdf-4d53-be79-db483078c06f"],
"x-runtime"=>["0.024586"],
"via"=>["1.1 google"],
"alt-svc"=>["h3=\":443\"; ma=86400"],
"cf-cache-status"=>["DYNAMIC"],
"report-to"=>["{\"endpoints\":[{\"url\":\"https:\\/\\/a.nel.cloudflare.com\\/report\\/v4?s=yZ%2BoWfPR%2FZ6y0S1F16wAPs3CQdCY5iY4GdhG2LQNHbkE%2FzeQ1r87oNY2mMEmzawGR1DgHHX8iFu3G956c8EAB18p9GGrDr%2FO0m9Vjw3%2BiXG6aDhwgVAEtOxB49qEXKxN7%2Fzo7LSGLGSbsvU%3D\"}],\"group\":\"cf-nel\",\"max_age\":604800}"],
"nel"=>["{\"success_fraction\":0,\"report_to\":\"cf-nel\",\"max_age\":604800}"],
"strict-transport-security"=>["max-age=15552000; includeSubDomains"],
"server"=>["cloudflare"],
"server-timing"=>["cfL4;desc=\"?proto=TCP&rtt=10623&min_rtt=10612&rtt_var=4002&sent=4&recv=6&lost=0&retrans=0&sent_bytes=2318&recv_bytes=1372&delivery_rate=263103&cwnd=187&unsent_bytes=0&cid=685ea18ed7c2e3d8&ts=191&x=0\"" ]
}
I also added logging of the chunk that zlib was trying to decompress, which showed that it is clearly plain text/not compressed. This leads to the code blowing up as it tries to decompress it, and that fails.
Inflation failed: incorrect header check, Compressed chunk: "{\"state\":\"no_longer_required\",
What are the steps to reproduce the issue?
Not really certain, make thousands of requests and some will come back with content-encoding: gzip but an uncompressed body.