We run a game mods CDN that pushes tens of thousands of different files daily, and up to 150TB of bandwidth monthly. Cloudflare caches anywhere from 85-90% of this traffic (which is awesome), but I’m looking to implement a downloads tracker via Cloudflare Workers so we can get some specific statistics on how frequently files are accessed, and how many times each file is downloaded.
Because we get traffic from all over the world, I don’t think Workers KV will work well for us, as we’ll be wanting to increment a download counter pretty frequently, and the eventual consistency model will almost definitely leave us with some traffic being “lost” as we have to read, increment, and then write back to Workers KV. We’d essentially need globally atomic operations, which Workers KV doesn’t seem to be suited for at this time.
So, I thought we could just make use of caching download numbers/times in global memory, and then send a request back to our server for database storage. However, due to the unpredictability of how some of these files are accessed (some constantly, and others only once or twice per day), I was wondering if there was any kind of event that a worker emits just before it’s about to shut down? This would give us the ability to cache download counts for as long as we could in memory, and then make a request to our server just before the worker completely exits. We could of course just make the request to our server every 10, 100, or 1000 requests, etc. but we’re trying to reduce the load on our origin as much as possible, as well as reduce the possibility of any traffic being “lost”. Does anyone have any suggestions for how best to approach this?