How is CPU time per request measured in Cloudflare workers


#1

On the workers resource limits page, I see the limits mentioned as,

An individual Worker script may consume up to:

  • 5-50 milliseconds CPU time per request, depending on Cloudflare Plan
  • 15 seconds real time per request
  • 128MB memory at any given time

As a developer, I’m curious to know how the worker platform measures and attributes CPU time for each request. Since there can be multiple requests running in parallel for the same request, I’m interested to know the implementation details.

I tried looking at various sources to see how it is implanted, I couldn’t find a reliable way to identify the CPU time per thread etc.

Can someone who knows the implemention details share this info?

Thanks,
JP


#2

I bet @KentonVarda knows.


#3

Hi @jayaprabhakar,

The workers runtime always knows which request it is executing on behalf of. For example, when a FetchEvent handler is first executed, obviously we know which request it was for. If the handler initiates an outgoing fetch(), we know that that subrequest is connected to the incoming request. When the subrequest completes and we go back to your FetchEvent handler, again, we still know what original request we are processing.

The operating system keeps track of how much CPU time has been used by each thread, so we can check that number before and after calling into your JavaScript to determine how much time was used by the worker. We can also ask the operating system to deliver a signal to the thread after, say, 50ms of CPU usage. If we don’t disable the timer before that, then the thread is interrupted to run a signal handler. In that signal handler, we can arrange to call V8’s TerminateExecution() function to cancel any JavaScript that was running. Hence, we can enforce a limit on how long the worker runs.


#4

Thanks @KentonVarda for the details. It was really helpful.

For other looking to implement something similar, these are the system calls.

Sample Code

import (
    "fmt"
    "golang.org/x/sys/unix"
    "syscall"
    "time"
    "unsafe"
    )
func main() {
    var start syscall.Timespec
    syscall.Syscall(syscall.SYS_CLOCK_GETTIME, unix.CLOCK_THREAD_CPUTIME_ID, uintptr(unsafe.Pointer(&start)), 0)
    fmt.Println("Thread CPU Time at start:", start);
}

#5

Uh, how is this useful on cloudflare workers?

EDIT: Or do you mean for making our own serverless chrome?


#6

Sorry, this was not related to workers. I apologize for using it as a stackoverflow.

I was curious to learn how it is implemented, I couldn’t think of the write keywords to search for this, until Kenton mentioned about OS tracking CPU time per thread, and then about timers by CPU usage and signal handlers.

So, once I found the exact system calls, I decided to share it back.


#7

No need to apologize, I found it really interesting but got a bit confused :slight_smile:


#8

I’m curious, what operating system are you running?


#9

Linux.

I should have named the specific syscalls in my previous comment but I wasn’t sure what kinds of details @jayaprabhakar was looking for. Sorry for making you search!