The performance for my workers at the 99th and 99.9th percentile seems to sometimes exceed 50ms and, according to the chart in the Cloudflare dashboard, even goes as high as 150ms. Even though that’s the case, I haven’t had a single “Exceeded resources” error.
After doing some performance testing with Workers to try and find the culprit (by just deploying differing versions and testing each one), I came across Metrics | Cloudflare Workers in the Cloudflare Workers docs. It states:
In some cases, higher quantiles may appear to exceed CPU time limits without generating invocation errors because of a mechanism in the Workers runtime that allows rollover CPU time for requests below the CPU limit.
It feels like this might be the case with my workers, but I’m not sure what this paragraph means.
What’s “rollover CPU time”? Does this mean that my workers are actually doing the work in <50ms?
It’s my understanding that if most of your requests consistently finish below the threshold, you build up some grace time (like rollover minutes on a cell plan) that can be used if an occasional request exceeds the threshold.
Ah! Well that makes perfect sense! Thanks, @john.spurlock! I’ll mark your answer as the solution as it seems a fitting explanation, though a note to consider for anyone viewing this from Cloudflare:
I’m currently testing the viability of Cloudflare Workers for a particular solution for a client. While the “rollover CPU time” is a blessing for anomalous requests, it’s still worrying for me that I can’t easily see how much of this time I’ve accrued.
The main concern for me here is that I’ll grow used to the sight of high-CPU-time requests in the 99th percentile and then one day it’ll catch me out and just stop working; I can’t see if I’ve got a good big budget of “rollover CPU time” left or whether I’m dancing on the line. I’m sure there are many considerations that go in to exposing a metric like that and I’m also sure you don’t want to encourage users to use that rollover time either, but thought I’d share my concern!
Thanks for the speedy response, @john.spurlock, and for the explanation over there in the other thread, @KentonVarda. Keep up the wonderful work and stay safe.