CF-Worker: Response closed due to connection limit

I am trying to wirte a CF-Worker. I am trying to create subrequests to an external service to collect information that shall be gathered and cached by the worker and returned to the user. I am requesting from the same domain with different parameters, not sure if this is relevant?

It seems that if I create more than 5 subrequests I get the following error response:

“Error: Response closed due to connection limit”

An example call would be: https://overpass2.kurviger.de/?fromLat=48.0&fromLon=8.0&toLat=49.0&toLon=8.4

Are there any limits in place that I couldn’t find in the documentation? I thought the limit would be 50 subrequests.

BTW: the script works if I run it locally with wrangler, but fails when being deployed.

This is documented here:

Cloudflare have said there’s no plans to change with this Workers Unbound too – which is a really pity as there are many examples of where this is useful (especially in GraphQL land where you often need a number of parallel resolvers)

1 Like

Thanks for the reply. This helped me get things sorted! I have seen that and thought I worked around this successfully, but I didn’t.

That’s what I did:

for (...);
  fetchArr.push(fetch(url, parameter));

  if(fetchArr.length >= 5){
    for (let index = 0; index < fetchArr.length; index++) {
      responses.push(await fetchArr[index]);
    }
    fetchArr = [];
  }
} 

Essentially I wait until the connection is closed before adding new fetches. But I missed that I have to gather the response as well, before the connection is actually closed.

I changed the code above as follows:

for (...);
  fetchArr.push(fetch(url, parameter));

  if(fetchArr.length >= 5){
    for (let index = 0; index < fetchArr.length; index++) {
      const response = await fetchArr[index];
      results.push(await gatherResponse(response));
    }
    fetchArr = [];
  }
}     

Now it’s working fine! It would be a lot better to perform these fetches all at once, but the current solution should be fine for now.

I marked @michael.hart.au’s response as the solution because he answered your question and deserves credit for that.

2 Likes