Worker generating twice as much requests as page views


#1

Hi,

I’ve enabled a worker on my site and setup different routes for it (enabled and disabled), but I noticed the worker is generating twice the amount of requests as page views. Theoretically, with the routes I’ve setup, the worker should only generate one request per page view. The reason I’m concerned is because I don’t want to exceed the 10M requests limit and I’m currently very close to it. But if I could have 1 request per page view like it should be, then my worker requests would be 50% lower and I wouldn’t have to worry about exceeding the limit.

Here’s my worker and the routes I’m using:

addEventListener('fetch', event => {
  let url = new URL(event.request.url)

  if (url.searchParams.has('fbclid'))
   url.searchParams.delete('fbclid')

  event.respondWith(
    fetch(url, event.request)
  )
})

ENABLED

*website.com/*

DISABLED

*website.com/cdn-cgi/*
*website.com/feed/*
*website.com/page/*
*website.com/category/*
*website.com/tag/*
*website.com/wordpress/*
*website.com/wp-json/*

Maybe there’s something I didn’t think about when creating these routes, but after examining all network requests to my site using Chrome inspector, these routes cover all the possible paths.

Any help would be greatly appreciated!

Thank you.


#2

:wave: @janvitos,

I believe workers will also count sub-requests. You might check your code against the conditions outlined in the FAQ here:

Also some additional information here on what the analytics tab shows/ counts:

-OG


#3

Thanks for the info!

I believe fetch(url, event.request) is generating a subrequest as well as the initial request. That would explain why it’s generating twice as many requests as page views. Does that make any sense? I thought only the initial request was counted towards the 10 million limit.


#4

:wave: @janvitos,

That’s not how I read the FAQ, but I’m unsure. Perhaps someone with more experience will clarify.

-OG


#5

Hi @janvitos,

That is correct – Workers are charged based on the number of inbound requests from visitors, not by the number of subrequests the script makes.

Regarding the issue that you’re experiencing, I can only speculate without knowing the details of how your site is implemented. A few common reasons:

  • Could it be possible that your origin is returning a redirect? Since your script’s fetch() call forwards the event.request object as the initializer dictionary, and event.request.redirect defaults to "manual", the script would pass redirects back to the visitor without following them, causing the number of inbound requests to double (or more). If so, I’d recommend finding a way to eliminate the redirect, or to explicitly force the worker to follow it with fetch(url, { ...event.request, redirect: "follow" }).

  • Does whatever HTML that gets served from website.com/* load a subresource (e.g., img tag) that also comes from website.com/*? If so, that subresource would necessarily also invoke your worker.

  • If your HTML does not set a custom favicon location (e.g., with <link rel="icon ...>), the browser may be fetching website.com/favicon.ico.

If those are definitely not the issue, then we’ll need more information about your site. I’d recommend raising a support ticket by emailing [email protected] from the email address on file with your Cloudflare account.

Harris

Edit: added favicon.ico case.


#6

Hi @harris , thanks for the detailed info!

Since you confirmed Cloudflare Workers are charged based on the number of inbound requests (and not the total of subrequests), then there’s no real issue here!

What I was referring to in my original post was actually the total number of subrequests that was roughly double the amount of pageviews. But as you confirmed, only the total number of visitor inbound requests are counted towards the 10M limit, am I right?

There are no redirects from my origin for the fetch() call.

My site does indeed load images and other assets for every visitor, but I’ve added “off” routes to disable the Worker on those assets (images, javascript, css, etc.). So theoretically, only the initial visitor request is making a hit on the Worker.

Now, another question for you. Where can I find stats about those inbound requests? Because when I go into the Workers section of the dashboard, I only see the total amount of subrequests. I’d really like to be able to see where I’m at regarding the 10M limit so I’m not overcharged (that’s basically what this post was all about in the first place).

Thanks!


#7

Jumping in the conversation.

When you say off routes, are you referring to disabled?


#8

That’s right! As you can see from my initial post, I enabled the worker everywhere and disabled it for every possible asset path. Basically, the router is disabled everywhere except for the main request (www.example.com/some-article/).


#9

My understanding, and I might be wrong, but disabling a route doesn’t exclude matches. It just makes the matches inactive.


#10

Interesting. Here’s what it says from the Cloudflare Workers doc:

Route patterns always participate in matching, even if they have no worker script associated with them. A route pattern with no associated worker script simply means, “Do not run a worker on this route.” For example, consider this pair of route patterns, one with a worker script and one without:

https://*.example.com/images/cat.png -> <no script>
https://*.example.com/images/*       -> worker-script

Like the example above, all HTTPS requests destined for a subhost of example.com and whose paths are prefixed by /images/ would be routed to worker-script , except for /images/cat.png , which would bypass Workers completely.

Thoretically, disabled routes “bypass Workers completely”. But are the initial requests to the disabled worker routes counted towards the 10M limit? That’s the tricky part.

If requests to disabled routes are indeed counted towards the 10M limit, then that means I need to figure out an enabled pattern that matches all post paths without using disabled routes. But since all my posts are all of the format www.example.com/some-post-name, I think it would be rather hard to find a pattern that can match all post paths, unless I’m missing something. I think I would then need to add a prefix to the URL, like www.example.com/post/some-post-name so I can use www.example.com/post/* for the pattern.


#11

This phenomenon happens to me when browsers call my workers with OPTIONS and GET generating two requests.


#12

To answer your last question (where can I find stats about inbound requests); if you look at the last (right most) tab on the Workers panel in the Dashboard, the one that says “Workers Status”, you can see the number of inbound successful and failed requests. Those requests are the ones that are billable to you.


#13

Requests to disabled routes are not counted as billable. Marking a route as disabled causes the system to bypass executing your worker when a request matches (assuming there’s no more specific route that matches), and you are only charged for requests where your worker executes.


#14

@alisman reminded me that another common cause for the double-request issue is favicon. You may want to disable the website.com/favicon.ico route, unless you’re already serving it from one of your currently disabled routes.

Harris


#15

Never thought about favicon! Awesome find.

Edit: My favicon was already being served from an “off” route. So I’m still trying to find the cause of the double inbound request per pageview.


#16

Thanks for the info. I’m still seeing roughly twice as many inbound requests as pageviews, so I guess I’m still missing an “off” route somewhere. But even after inspecting every single network request through Chrome inspector, I still cannot find the culprit. And parsing my server’s log file does not show anything other than what’s already covered by the “off” routes. So is it possible that the Cloudflare Worker inheretently / mistakenly records two (2) requests instead of one (1)?

Thanks for the clarification.


#17

Hi @janvitos,

I think to debug this further you will need to file a support ticket. You can do so by emailing [email protected] from the email address registered with your account – I’d suggest including a link to this thread. Doing so will allow us to look at your account configuration in more detail and figure out what’s going wrong.

Harris


#18

Excellent Harris, I will do so. Thank you!


#19

Hi @janvitos,

When I load your site, my browser fetches /favicon-32x32.png and /favicon-16x16.png, neither of which is covered by your filters, so they would invoke Workers requests. Could this be the culprit?

If that’s not the problem, could you please describe how you are measuring “page views”, and how you are measuring inbound requests?

Note that there is no 10M “limit”. If you exceed 10M requests in a billing cycle, you’ll simply be charged for additional requests, at a rate of $0.50 per million.


#20

Hi @KentonVarda, thanks for your help.

Indeed, I just changed the favicon files today and forgot to add those files to the OFF routes, so I will add those now. Though, before today, the favicon was covered and ignored by Workers since it was under the /wordpress/* OFF route, but I was still getting roughly twice as much inbound requests as page views.

I’m measuring page views through Google Analytics and Google AdSense which give me roughly the same number every day. Inbound Worker requests are always roughly double that number.

I know there’s no 10M limit, but I’d rather be able to fix the Workers problem at the source in case one day I do go over the limit. If I can get the Worker inbound requests down to roughly the same amount of page views like it should be, then I’ll never have to worry about hitting the limit and I’ll have peace of mind.

I opened a ticket with Cloudflare and it was escalated to the Workers team. Hopefully they’ll be able to figure out what’s going on. Like I mentioned before, I’ve already gone through every possible network request & server log file to find what could cause the extra Worker requests, but to no avail.

If you have any other suggestions, I’ll be happy to try them out!

Thanks again for your help.