Workers and sub-requests

Hello,

Some quick questions about workers that hopefully someone can help with and I haven’t been quite clear on:

  1. I came across this post:

"There is a not well documented rule that rate limits workers with subrequests. Mostly API users see this.

If you have an API user that is doing more than 2,000 requests a minute to your api for the same colo, zone and “eyeball IP” (and each of those is issuing a subrequest via your worker) you’ll be rate limited. The rate limit is basically 2k subrequests per minute per colo / zone / IP. Once you hit that you start serving 429s."

(… from Massive rate-limiting issues with Worker in production)

Just to clarify that, as it is not documented, does that mean every request sees a 429 beyond that limit, or only the ones from that same user (so if someone is making lots of requests from, say, 1.2.3.4, they see a 429 for the 2001st, but someone else, say, from 5.6.7.8 sees no issue)

  1. Does a request to KV count towards that 2000, so is that considered a sub-request?

  2. Are requests to worker’s Cache API rate-limited the same as if calling api.cloudflare.com externally? I see direct calls to the API are rate-limited to 1,200 per 5 minutes. So like, if a worker deletes a cached entry using its built-in Cache API, does that count towards it?

Thanks!

Hi @gregsanderson,

Our exact abuse protection measures are not documented for a few reasons:

  1. They may change frequently and without warning.
  2. The details can be subtle and complicated.
  3. Knowledge of the exact measures might make it easier for abusers to work around them.
  4. We don’t intend for developers to have to design around these measures; instead, we will adjust the measures to match your needs.

If you observe error 1015 (HTTP status 429) in practice, or you are worried that you might, please raise a support ticket and we’ll be happy to adjust the anti-abuse measures so that they do not impact your use case (as long as your use case isn’t abusive, of course!).

Are requests to worker’s Cache API rate-limited the same as if calling api.cloudflare.com externally?

No, the Cache API and api.cloudflare.com are completely different things. The Cache API operates on the edge, only affecting the colo where it is called. api.cloudflare.com is a centralized service that manages configuration for your site. In general, nothing that a Worker does when executing on the edge will interact with api.cloudflare.com – unless you actually make a fetch() request to api.cloudflare.com, of course. :slight_smile:

8 Likes

Hi @KentonVarda,

Thanks for getting back to me so quickly and for the clarification on that. Makes sense.

:slight_smile: