Worker generating twice as much requests as page views


#21

This is likely a fault of users running ad/content blockers.

Many ad/content blockers (including Chrome itself if your page violates the better ads initiative) block Google Analytics/Adsense since it’s believed Google uses it to track users around the web, and if that happens then Google Analytics won’t get a page hit.


#22

So you’re saying I might be actually getting twice more page views on my website than what’s being recorded in Google Analytics & AdSense because of ad blockers? Isn’t that a bit high of a number? 85% of my traffic is from mobile / tablets and most of these devices aren’t using any ad block mechanism. I’m a bit skeptical about ad blockers being the main cause, but your hypothesis is interesting nevertheless. I guess ad block might still have somewhat of an impact that I didn’t count in.


#23

Search engine bots, might be another reason.
We have a client that has 10M urls, every day Google peaks traffic around 6 am UTC.
This will count towards the workers’ bill.


#24

Interesting find @adaptive! And I assume search engine bot visits are NOT recorded as page views, but are still counted as inbound Worker requests. But in my case, I rarely have more than 10,000 search engine bot visits per day, so it couldn’t be the main cause of the double requests. But it might still inflate that number a bit.


#25

Here is how I implement the logging mechanism on the workers.
You will find what is going on.

Try something like this:


#26

Thanks!


#27

Hi @janvitos,

I’m measuring page views through Google Analytics

As some others have noted, Google Analytics undercounts by a lot. GA depends on the client executing some JavaScript that performs an additional HTTP request to analytics servers – if that doesn’t happen for any reason, then the visit doesn’t get counted. Possible reasons include ad blockers, or because the client is a crawler (not a real user), or because the user started loading the site but didn’t finish, maybe because their connection was too slow, or many other reasons. The rate at which GA undercounts can vary a lot between sites, but from my understanding 50% is an entirely plausible fraction to miss.

Cloudflare’s analytics, meanwhile, are much more accurate, because we count the actual requests that served the original content; we don’t depend on the client doing anything extra.

If you want to investigate further, I’d suggest doing something like @adaptive suggests where you count requests in your worker itself. I bet you could even post directly to the Google Analytics API from your worker – unblockable analytics!


#28

Well it does make sense that a lot of requests are not being counter towards page views. I just didn’t think it would make up for 50% of the total trafic. But I guess it’s possible.

I totally understand that, but why doesn’t Cloudflare analytics include page views? I would definitely like to see what Cloudflare is recording vs what Google Analytics is recording. It would be an interesting metric to have.

That sounds like a great idea. I took a quick look at his Worker script and it’s a great way to log & debug Worker requests. I will surely try to implement something with Google Analytics and see if I can get more accurate metrics that way.

Thank you very much for your help!


#29

I can imagine this is because it’s harder that it seems server side. You would need to count all files passing through that match a specific type, but what do you include? HTML for sure, PHP (maybe?), txt (yes? no?), it’s not a simple question and it requires more processing server side.


#30

That makes total sense. And that’s why using Google Analytics with a piece of code on every page makes the task much easier. Then GA just records everything that passes through and records a page view for every hit (basically). But I guess it’s not so precise in the end.


#31

Yeah, unfortunately between AD blockers, JS disabled, bots it’s hard to have all traffic accounted for.