Workers response bodies slow + 0 concurrency on certain POPs

I am observing slow responses from Workers only on certain POPs, related to streaming a response body to the browser, and am looking for some insight into why this is happening.

Code

Below is a 11 line Workers script, nothing else is required to reproduce this.

The worker returns a HTML page for / that simply hits the worker again 5 times at a different URL, starting 5 of the same requests at the same time. For those URLs the worker proxies the request to the SOME_URL asset, which is a few hundred KB.

the value of SOME_URL is any internet asset - I used the 450 KB PNG image on the Cloudflare homepage, but I have observed this issue with non-image assets, so I don’t think any image optimization going on is relevant.

const INDEX = `<script>for (let i = 0; i < 5; i++) fetch("/other");</script>`;
const SOME_URL = "https://cf-assets.www.cloudflare.com/slt3lc6tev37/76x52jIsr93tqZq0h3HCFW/99238ae9f872014e19514793c2899208/Blueprint_after_ConnectivityCloud_Resized.png";
export default {
	async fetch(request, env, ctx) {
		if ((new URL(request.url)).pathname === "/") {
			return new Response(INDEX,{headers:{'Content-Type':'text/html'}});
		}
		return fetch(SOME_URL);
	},
};

Observations

My internet connection hits the LAX POP, sometimes SJC. When I’m lucky enough to reach the LAX POP I can see that the five simultaneous requests stream their responses concurrently (overlapping blue bars) and completes in ~500ms. (See later screenshot)

When I’m unlucky and get the SJC POP the response times are long, with most requests finishing in serial and very constrained bandwidth, taking ~25 seconds to finish all requests:

If I simply await and consume the response body, without returning it to the browser, by changing the last two lines:

const res = await fetch(SOME_URL);
await res.arrayBuffer();
return new Response("Hello World");

I have short roundtrips even when connecting to SJC (~200 ms).

I can reproduce this very consistently as long as that specific POP is assigned. So that eliminates origin latency as the root of the problem.

Questions

The issue seems isolated to a specific POP and streaming semi-large response bodies to the browser, which causes all requests to that specific Worker to back up.

  • Is there a problem in the code somewhere, or is this a limitation of the Workers platform?
  • Is this related to some part of the CF stack a the account level (rate limiting, DDOS prevention, etc?)
  • Is this related to network prioritization, and can it be mitigated with higher-tier plans?

Image of LAX response times including proxied asset body:

Image of SJC response times when consuming the body in the worker but not streaming it to the browser: