How do HTTP range requests work with Workers?

I’m experimenting with Workers by serving some audio file from a KV.

const file = await AUDIO_FILES.get(key, 'arrayBuffer');

const response = new Response(file, {
	status: 200,
	headers: {
		'Accept-Ranges': 'bytes',
		'Content-Type': 'audio/mpeg'
	}
});

return response;

I’m forced to add the Accept-Ranges: bytes on the Worker response, otherwise Chrome won’t let me change the currentTime of the audio element.

I can see on the dev tools that Chrome is not downloading the full file when pressing play, but as far as I can tell, my worker has completed its job.

Are the audio bytes cached somewhere in Cloudflare’s network?

Is the worker actually still serving the bytes for further requests of different ranges? How could I confirm this? If this is the case, how long will the worker keep serving bytes?

I haven’t been able to figure out how range requests work with Workers… but I’ve been able to send the full audio file to Chrome and fix the currentTime problem.

First I’m setting these headers on the Worker response:

const file = await AUDIO_FILES.get(key, 'arrayBuffer');

const response = new Response(file, {
	status: 200,
	headers: {
		'Accept-Ranges': 'none',
		'Content-Length': file.length,
		'Content-Type': 'audio/mpeg'
	}
});

return response;

Then, instead of simply using the audio URL on the audio.src, I needed to to use a MediaSource object:

const mediaSource = new MediaSource();
audio.src = URL.createObjectURL(mediaSource);
mediaSource.addEventListener('sourceopen', sourceOpen, { once: true });

function sourceOpen() {
	URL.revokeObjectURL(audio.src);
	const sourceBuffer = mediaSource.addSourceBuffer('audio/mpeg');

	fetch(audioUrl)
	.then(response => response.arrayBuffer())
	.then(data => {
		sourceBuffer.appendBuffer(data);
	})
	.catch(error => {
		console.log(error);
	});
}

Finally, to fix a weird bug in Safari which caused a 1 second delay when setting the currentTime, I needed to initialize an audio context object anywhere in my application:

const AudioContext = window.AudioContext || window.webkitAudioContext;
const audioContext = new AudioContext();

Still, I would very much like to know how range requests work if anyone from Cloudflare has the answer. :slight_smile:

This doesn’t address you problem specific, but addresses byte serving in 2021 generically.

With endless amounts of 2010s middleware (“layer 7 routers”) being peddled for the last 10 years, practically all HTTP responses are “chunked encoding” for no reason, on CF and other cloud providers.

IIRC CF from cache delivers gzip not chunked with content-length header. CF from origin is always chunked I think, gzip browser optional, even if origin returns Content Length header. These forums say CF always (or 99%) gzip decompresses and recompresses from origin. AKA your Zopfli vs stock gzip pre-compressed assets on origin are almost pointless unless you zopfli an actual .zip or .png or tar.gz disk file and not nginix or apache precompress.

Content-Length/“not chunked”/range has always been a weak point of CF Content-Length header on fetch requests getting omitted - #2 by adaptive

If the file is produced sector by sector (KB by KB), lets say a SQL query to PDF generator, or SQL to JSON, yes, its really unknown how many pages/rows or what final contents length are. But for disk files, its sad when you can’t save client bandwidth by doing range to just get the metadata/header of some image, or video, or 1 attribute out of a massive XML file you know will 99% of time live in the 1st KB of the XML file.

I can’t remember if I did the experiment with Github/Fastly HTTP server or CF HTTP server, but inside XHR inside Chrome, since fetch() has NO WAY to cancel the transfer, I called abort() after 1-4 KB when I got the first 1KB chunk inside blob->ReadableStream. After my RST packet was sent from Chrome, CF or Fastly, ACKed my RST packet, but hilariously sent another 20 KB of data down the wire to me before remote server stopped sending me packets (it didn’t stream to the end). Does ATT/TMobile/Verizon count TCP packets sent after RST/FIN as part of your cellphone’s transfer limits? lol

If only I had Range on a JSON response I would be in heaven. How to abort a transfer on HTTP 1.1 without RST/FIN the socket? not possible :slightly_frowning_face:

client/eyeballs and CFW fetch API has NO WAY to terminate a file transfer once started. HTTP 2 has NO WAY to terminate a file transfer once started AFAIK. Range and Content-Length is barely supported nowadays on the internet. The parallel file downloaders/accelerators of the mid 2000s minimally work anymore, its sad. aria2c tool can max out my cable modem with parallel downloading, but non-CF CDN servers never pump at max speed.

Even worse some servers return 400 or 405 for HEAD nowadays.

I just wanted 200 (show localStorage to user) or 401/403/429 (token expired, draw auth UI ASAP since we have WAN and dont show localStorage to user) or 0 (no WAN, show localStorage to user, UI & localStorage will keep user busy until he walks back onto the cell network or his phone finds the next wifi AP in the office building). I dont need the body. But NOOO, HEAD doesn’t work, and the smallest JSON endpoint is 20 KB of customer root billing info. I ultimately wrote a CFW return the HTTP status code instantly in a new response object and never read the origin response body object. The origin API is MVC/XSLT-ish. Every POST or GET API call returns the 20 KB of root billing info over the wire in JSON.

Twitter (or youtube?) invented HLS just because “influential” devs and influential infrastructure companies abandoned HTTP 206/range/byte serving infrastructure. Even though I’d rather have my eyeballs connect to not-CF origin and save on my CFW quotas. I shave down mega-responses with CFWs now all the time, to keep all text based responses under 1300 bytes uncompressed, and definitely “1 packet” response after gzip. If the user opens his detailed profile, or I know I need a mega sized (>1 packet) origin response, I have client directly connect to origin server with CORS and the data transfer is what it is. But “zip code”, “shipping type” (integer) estimates, needs to return a 5 character string, not 20 KB of garbage including one off “integrity audits” by random security consultant firms and financial auditors, that the customer is “safe and secure” and not on a fraud/credit score/embargos/sanctions/terrorist watchlist. Why that info goes over the wire, I dont know. The fields are undocumented by the 3rd party integration API docs they supply. I integrate this ecommerce system with something else. For a mobile app, 35-500 KB a tap XHR JSON responses are criminal.

Ive also learned that on CF specifically, if my endpoint, in my worker, always returns less than 100 byte ASCII HTTP bodies, I always put "cache-control":"no-transform" to disable GZIP and chunked. GZIP and chunked add 15-45 bytes more response body on HTTP 1.1 clear and 1 extra packet (8 byte body TCP packet, empty chunk trailer). On CF HTTP2, I’ll get a HEADER and DATA frame same packet if no-transform, otherwise its 3 SMALL TCP packets, HEADER packet, DATA with body packet, DATA end of stream flag packet.

1 Like

Thanks for all the info. Super interesting stuff.

I didn’t update this thread a couple of days ago, but I was able to implement range requests with a worker and stream chunks to the audio player with the bytes coming from KV.

The issue is that for every chunk I consume 3 reqs (OPTIONS + GET + KV). If my chunks are say 100kB then for a 5MB MP3 I consume 150 reqs to play that file which becomes expensive at scale.