Workers KV, Response Object Inadequate

Unless I am missing something, at scale it would seem the current Response object is not up to the task of handling a large result set from a KV store. The current Response object model, being based on the browser model, requires all the results to be assembled into body content before the Response object is created. The body is not a writable stream. It would be useful if Response also supported a send and and end methods so results could be streamed. One possible approach would be to treat the body as a WritableStream until end is called and if end is not called prior to the Worker giving up control automatically call end.

Are we talking about a situation where you are trying to read a large object from KV, or one where you are writing a large object to KV? This is from inside a Worker right?

I am talking about the possibility of retrieving hundreds of objects from KV and “streaming” them back to the browser. Breaking a single large object into multiple sends would be pretty clunky on the client side. However, the body property of a Response object in the browser is a proper ReadableStream, so writing a reader that handles line delimited sends would be trivial. With send capability inside a Worker I could, for example, send each object separately and respond to it with a callback or even process with an async generator written around the ReadableStream. And yes, inside a Worker,

Why not using TransformStream for streaming Response? Examples here.

1 Like