Disable request buffering

I would like to disable request buffering, I have spoke on discord and filed an issue. My problem as explained is Cloudflare is buffering my request, I have a long running request that I wish to read in chunks as they arrive using a BYOB stream reader, it seems to only work if the client signals an end of stream, at that moment cloudflare starts processing my request.

I was advised that I should disable request buffering.

The discord public chat thread is here Discord

This is Workers, right? Can you edit the title of your post (category, actually) so it’s in Developers → Workers? That should get more attention. @matteo does quite a bit in Workers, as does @thomas4.

1 Like

I believe this was solved with a page rule that forced no cache.

2 Likes

anyway to test this pagerule or know what it is?

Yes, Cache Level: Bypass

1 Like

I have set this on live2.zod.tv

If you visit live2.zod.tv it serves you a simple websocket, the websocket echos the chunks of a chunked encoding upload.

You can test slowly pushing data by using

#!/bin/iex
post_req = """
POST / HTTP/1.1
Host: view2.zod.tv
Accept: */*
Transfer-Encoding: chunked\n
"""

{:ok, socket} = :gen_tcp.connect('view2.zod.tv', 80, [])
:ok = :gen_tcp.send(socket, post_req)

:ok = :gen_tcp.send(socket, "fff4\r\n")
:ok = :gen_tcp.send(socket, :crypto.strong_rand_bytes(65524)<>"\r\n")

#We need to send this to get cloudflare to proc the entire stream
:ok = :gen_tcp.send(socket, "0\r\n")
:ok = :gen_tcp.send(socket, "\r\n")

In short the proposed solution to turn ‘cache level: bypass’ does not work.

I haven’t tested Websockets at all yet and streaming chunks of data is still very much undocumented. There’s just handful of posts here on the forum too. Sorry I can’t be of much help until I have time to work on it.

1 Like

Okay np, thanks for looking at it regardless, also the websockets are not part of the equation, they work fine i think because I testing with fetching to another DO (instead of echoing to websockets) and the same blocking occurs.

The problematic part is chunk encoding / body streaming.

I’ll let you know if I ever get assigned to streaming, I know it’s on our roadmap (where I work).

1 Like