Cloudflare worker fetch ignores byte request range on initial request

As per this discussion How do you do the equivalent of the page rule cache_level: "bypass" in a worker fetch?
I am also trying to get and return a byte range from a large file and I also have the problem that every fetch request in the worker for the large file returns the entire large file not the byte range requested.
I am happy for Cloudflare to retrieve and cache the entire file but expected behavior would be that the initial request still returns the byte range requested rather than the entire file as per Albert’s response:
“When a client specifies the Range header in a request for a cacheable resource, Cloudflare fetches the entire resource and serves the specified range”
Unfortunately I can confirm it is not serving the specified range until the request is made a second time.

This means that executing the exact same request twice results in different outputs each time.

For example here’s a standard request to a large 5Gb tar file

curl -s -D - -o /dev/null GET https://f004.backblazeb2.com/file/gc-test-public/a_large_tar_file_5Gb.tar
HTTP/1.1 200 
x-bz-file-name: a_large_tar_file_5Gb.tar
x-bz-file-id: 4_z4cd6dadda64893e173e30513_f2000d66df7d808ce_d20220627_m232055_c004_v0402004_t0005_u01656372055525
x-bz-content-sha1: none
X-Bz-Upload-Timestamp: 1656372055525
Accept-Ranges: bytes
Content-Type: application/x-tar
Content-Length: 4980920320
Date: Mon, 27 Jun 2022 23:30:42 GMT


Adding in the byte range header returns the specified range as expected

curl -s -D - -o /dev/null GET https://f004.backblazeb2.com/file/gc-test-public/a_large_tar_file_5Gb.tar -H 'range: bytes=0-512'
HTTP/1.1 206 
x-bz-file-name: a_large_tar_file_5Gb.tar
x-bz-file-id: 4_z4cd6dadda64893e173e30513_f2000d66df7d808ce_d20220627_m232055_c004_v0402004_t0005_u01656372055525
x-bz-content-sha1: none
X-Bz-Upload-Timestamp: 1656372055525
Accept-Ranges: bytes
Content-Range: bytes 0-512/4980920320
Content-Type: application/x-tar
Content-Length: 513
Date: Mon, 27 Jun 2022 23:31:07 GMT

Here’s a minimal Cloudflare worker to request the same file with the same byte range

export default {
  async fetch(request) {
    const response = fetch('https://f004.backblazeb2.com/file/gc-test-public/a_large_tar_file_5Gb.tar', {
      headers: {
        'Range': 'bytes=0-512',
      }
    })
    return response;
  },
};

Running this worker under wranger dev the first request ignores the byte range request and returns the full file:

curl -s -D - -o /dev/null GET http://localhost:60337/
HTTP/1.1 200 OK
date: Mon, 27 Jun 2022 23:31:59 GMT
content-type: application/x-tar
content-length: 4980920320
cf-ray: 7221fab02eefa947-SYD
last-modified: Mon, 27 Jun 2022 23:31:58 GMT
cf-cache-status: MISS
expect-ct: max-age=604800, report-uri="https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct"
x-bz-content-sha1: none
x-bz-file-id: 4_z4cd6dadda64893e173e30513_f2000d66df7d808ce_d20220627_m232055_c004_v0402004_t0005_u01656372055525
x-bz-file-name: a_large_tar_file_5Gb.tar
x-bz-upload-timestamp: 1656372055525
report-to: {"endpoints":[{"url":"https:\/\/a.nel.cloudflare.com\/report\/v3?s=u7pb3WNFXzztmZGugWayEg%2BlMZWVn3SpuiHJr430EDKH5Zz2Kh0Nfgue3jHmCguYTOfRwK0p8KZnDq8palFZHaNCGUdFDDE3D0UvnRi%2Fc27LeH9HNMkk4P0DaWfM427rT7t65oqBdKlHxeAmuqPoTkmJSA%2B15TyNKw%3D%3D"}],"group":"cf-nel","max_age":604800}
nel: {"success_fraction":0,"report_to":"cf-nel","max_age":604800}
vary: Accept-Encoding
alt-svc: h3=":443"; ma=86400, h3-29=":443"; ma=86400
server: cloudflare
Connection: keep-alive
Keep-Alive: timeout=5

A second request made immediately afterwards returns the byte range as requested:

curl -s -D - -o /dev/null GET http://localhost:60337/
HTTP/1.1 206 Partial Content
date: Mon, 27 Jun 2022 23:32:04 GMT
content-type: application/x-tar
content-length: 513
cf-ray: 7221fad2ee2ea947-SYD
content-range: bytes 0-512/4980920320
cf-cache-status: MISS
expect-ct: max-age=604800, report-uri="https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct"
x-bz-content-sha1: none
x-bz-file-id: 4_z4cd6dadda64893e173e30513_f2000d66df7d808ce_d20220627_m232055_c004_v0402004_t0005_u01656372055525
x-bz-file-name: a_large_tar_file_5Gb.tar
x-bz-upload-timestamp: 1656372055525
report-to: {"endpoints":[{"url":"https:\/\/a.nel.cloudflare.com\/report\/v3?s=hCTu5i7Zm%2F46RIlJJuEMaA2vTpDujffC4Se9OXUmEheQmTpYxuwHVM7l7PBwSojwYCI%2BdNhTf009dCamxlRc2Avfmkb0GuVMB94htqEvGAiUD13XHX%2FR73zwxwcsISOJkZ04Bv4DrL5Lpch29Dir96ZKHJk7cvV8xw%3D%3D"}],"group":"cf-nel","max_age":604800}
nel: {"success_fraction":0,"report_to":"cf-nel","max_age":604800}
vary: Accept-Encoding
alt-svc: h3=":443"; ma=86400, h3-29=":443"; ma=86400
server: cloudflare
Connection: keep-alive
Keep-Alive: timeout=5

I can confirm this also happens on production and is not an issue with wrangler.

So far I have only managed to reproduce this issue with large files. Attempting this with, for example, a smaller image file does result in the byte range being returned the first time. I suspect that for smaller files Cloudflare is requesting the full file but can load and cache it and return the byte range requested within some timeout period.

I consider this to be a bug with the Cloudflare worker fetch request as the exact same worker fetch request is returning different results depending on when it is called.

Turing off caching for the requested domain as detailed here: How do you do the equivalent of the page rule cache_level: "bypass" in a worker fetch? does resolve the issue as does aborting the initial fetch request and immediately re-issuing the same request. Neither of these solutions are ideal so it would be good to see this bug fixed.

Please note that the file I have used in the example above has been moved.