Outbound traffic receives response with 429 status code

Hello,

We are load testing our worker implementation and found a weird behavior.

This worker internally call Cosmos to retrieve some data using @cfworker/cosmos. Everything works quite well under a light load such as 50 RPS. When the load bumped to 500 RPS, we found that all Cosmos requests are getting responses with status code 429. After inspecting the response, it is not Azure Cosmos response style and it is lack of all Azure style headers.

This issue would last for around 2-3 minutes even for a single request, then it is back to normal. We could not find any 429 response records in the Azure dashboard and we could still communicate with Azure Cosmos using other clients.

We are suspecting there might some outbound traffic rate-limiting for in Cloudflare workers. But could not find any document detailing that.

If anyone has a similar problem, could you please share more information with me?

Thanks!

Yuan

1 Like

AFAIK Cloudflare would not rate-limit by showing a 429, rather if you went over the 50 subrequest per request limit it would throw an error. However, at 500 RPS this might trigger some deep, internal abuse-prevention mechanism, so I’d recommend you create a support ticket at https://support.cloudflare.com/hc/en-us/requests/new. My only other theory on this is that it could be the Azure Cosmos’s service rate limiting your “IP” (Cloudflare) at an early load balancer and not your general account, however, this might also be hard to confirm.

2 Likes

same link https://developers.cloudflare.com/workers/about/limits/#burst-rate-limit suggests for non paid worker plans

Burst Rate Limit

Accounts using the Workers free plan are subject to a burst rate limit of 1000 requests per minute. Users visiting a rate limited site will receive a Cloudflare 1015 error page. However if you are calling your script programmatically, you can detect the rate limit page and handle it yourself by looking for HTTP status code 429.

1 Like

Thank you for the info. Actually, we are using the paid plan and we are still waiting for a response from Cloudflare. Meanwhile, I think there might be someone else experiencing a similar issue if there was really some rate-limiting internally, that is the reason I also post the question in the community.

That is all good points. And agree, they are all hard to confirm …