Streaming responses

I’ve been trying to emulate a slow connection by streaming responses from a worker. As you can see in the following snippet, in theory each line in the call response should be sent individually but in practice the response is buffered and sent as a single response. Is there a way to enable streaming in a case like this?

addEventListener('fetch', event => {
  event.respondWith(fetchAndStream(event.request))
})

const char = '.';
const len = 10;

const pause = ms => new Promise(resolve => setTimeout(resolve, ms));

async function fetchAndStream(request) {
  let { readable, writable } = new TransformStream();
  const defaultWriter = writable.getWriter();
  const encoder = new TextEncoder();
  Array.from({ length: len }).reduce(async (prev, _, i) => {
    await prev;
    await pause(100);
    defaultWriter.write(encoder.encode(`${Math.random()}\n`, { stream: true }));
    if (i === len - 1) { defaultWriter.close() }
  });
  return new Response(readable, { status: 200 });
}

Streamed responses work differently -

I believe the requirement for a response to be sent in chunks is your async response function must call another async function (without awaiting it) and return the response with body being the readable stream. This function will continue to run - as if it was a event.waitUntil (although you don’t use that here) - and can continue to push data to the response regardless of the 5ms-50ms worker time limit.

Here’s another implementation example

It might not be obvious from the example, but most of the body of the code is not being awaited, in the following example, “here” is the first thing to get printed in the console, then you get the 1, 2, 3…

addEventListener('fetch', event => {
  event.respondWith(fetchAndStream(event.request))
})

const char = '.';
const len = 10;

const pause = ms => new Promise(resolve => setTimeout(resolve, ms));

function fetchAndStream(request) {
  let { readable, writable } = new TransformStream();
  const defaultWriter = writable.getWriter();
  const encoder = new TextEncoder();
  Array.from({ length: len }).reduce(async (prev, _, i) => {
    await prev;
    await pause(100);
    console.log(i); // this is printed in sequence, separated by 100ms
    defaultWriter.write(encoder.encode(`${Math.random()}\n`, { stream: true }));
    if (i === len - 1) { defaultWriter.close() }
  });
  console.log('here'); // this is printed before anything else
  return new Response(readable, { status: 200 });
}

So I’m wondering if it might have to do with the “releaseLock” or if CF simply buffers smaller responses.

Just tried a different thing, and yes, the problem was that CF buffers responses when they’re really small. Once I tried sending batches of 1000 character long strings I can see the streaming response.