Cache revalidation race condition results in HTTP stream failure

I’m able to consistently reproduce a race condition that seems to be caused by an issue in the way Cloudflare is handling parallel requests for cacheable resources that require revalidation.

I’ve uploaded a 100MB test file to https://www.neilpomerleau.com/files/100MB.zip and configured the origin server to return a response header of cache-control: public, no-cache, must-revalidate in order to require cache revalidation, per the Cloudflare documentation. I’ve also created a “Cache Level: Cache Everything” page rule for this URL to ensure it is considered for the edge cache.

The client needs to perform a HEAD request to determine the size of the content, immediately followed by a GET request to retreive the content. Cloudflare returns a valid response to the HEAD request with a cf-cache-status: MISS header, implying that Cloudflare is now warming the edge cache. However, the subsequent GET request is terminated abruptly after returning a cf-cache-status: REVALIDATED header.

The subsequent request completes correctly if revalidation is not immediately required, either by changing the response header to cache-control: public, max-age=0, s-maxage=5, must-revalidate to skip revalidation or waiting 5 seconds to perform the GET request later with revalidation. It is also mitigated if the content is treated as dynamic and always streamed from the origin, i.e. disabling the “Cache Level: Cache Everything” page rule.

This can be reproduced with the following bash script (a random query string is appended to the URL to ensure a cache miss):

#!/bin/sh

TEST_URL="https://www.neilpomerleau.com/files/100MB.zip?${RANDOM}"
curl -v --head "${TEST_URL}" && curl -v "${TEST_URL}" -o /dev/null

Which gives the following output after printing the headers:

{ [634 bytes data]
* HTTP/2 stream 1 was not closed cleanly: INTERNAL_ERROR (err 2)
  0  100M    0  254k    0     0  1329k      0  0:01:16 --:--:--  0:01:16 1331k
* Closing connection 0
* TLSv1.2 (OUT), TLS alert, Client hello (1):
} [2 bytes data]
curl: (92) HTTP/2 stream 1 was not closed cleanly: INTERNAL_ERROR (err 2)

This topic was automatically closed 5 days after the last reply. New replies are no longer allowed.