How do workers combine with page rules?

Developers Workers
,
#1

Continuing the discussion from Cloudflare Workers Beta Feedback:

It’s unclear to me how Workers work in relation to Page Rules.

I ran into a situation today whereby a Page Rule was not applied to the request handled by a Worker.

The request matches this Page Rule:

  • https://example.com/*: cache level: cache everything, browser cache: 1 hour, edge cache: 1 month.

With that rule Cloudflare responds for my HTML with:

cache-control: public, max-age=3600

But for the URL that I’m beta testing my worker on, no cache-control header is added to the HTML response. Why is that?

My worker does perform a request to another domain, but if the browser visits https://example.com/my-page/, shouldn’t the Page Rule for that URL kick in? But in my case the Worker actually disables the Page Rule.

I understand from the following quote that Workers cannot set cache-control headers. That makes it more important for me that Page Rules do work, because otherwise there’s no caching happening with the Worker.

Setting browser cache in a worker
#2

Hi @JuM,

When it comes to combining other Cloudflare features with Workers, some features run “before” Workers (they run on the request before it reaches your Worker, or they run on the response returned by your Worker) while other features run “after” Workers (they operate on subrequests (fetch()es) made from your worker). (Note that the words “before” and “after” here are referring to where the feature sits in the pipeline between the user and your origin, not strictly to time of execution. Hence a feature that is “before” Workers, but operates on a response, actually executes after your worker has completed. Maybe it would be better to use the words “in front of” and “behind”…)

As a rule of thumb, “security” features (WAF, DDoS protection, CAPTHAs, etc.) run before Workers, while other features run after.

When it comes to Page Rules, whether the rule applies before or after Workers depends on what feature it is affecting.

In your case, you have set a Page Rule affecting caching. Caching runs after Workers. So, your page rule applies only to subrequests. But, it sounds like your subrequests are being pointed to a different domain, not your own. So, those subrequests do not match your page rule, since your page rule can only match your own domain.

Instead of using page rules, you can get the behavior you want by doing something like:

// Fetch the remote URL with "cache everything" and edge cache TTL of 1 month.
let response = await fetch(request, {cf: {cacheTtl: 2592000}})

// Clone response object so that we can modify it.
response = new Respnose(response.body, response)

// Set browser cache TTL of 1 hour.
response.headers.set("Cache-Control", "public, max-age=3600")

This achieves exactly what you described as your page rule, without a page rule.

4 Likes
Browser Caching Issues
#3

Thanks for the thoughtful and thorough reply! :slight_smile: I’ve implemented the approach you outlined in my Worker and it seems to be working fine.

One thing that confuses me a bit is that the expires header that’s automatically added by Cloudflare is always off when a Worker sets the Cache-Control header. Is that expected behaviour?

For example:

cache-control: public, immutable, max-age=2700
cf-cache-status: HIT
cf-ray: 42bc2b69dad055ac-ORD
content-security-policy: upgrade-insecure-requests
content-type: text/html; charset=utf-8
date: Sat, 16 Jun 2018 09:15:02 GMT
expires: Sat, 16 Jun 2018 10:15:02 GMT
server: Cloudflare

2,700 seconds added to 9:15:02 doesn’t give me a time of 10:15:02.

Or this:

HTTP/2.0 200 OK
date: Sat, 16 Jun 2018 09:11:43 GMT
content-type: text/html; charset=utf-8
cache-control: public, immutable, max-age=2940
cf-cache-status: HIT
cf-ray: 42bc268fcf069cb3-AMS
expires: Sun, 17 Jun 2018 09:11:43 GMT

Here the expires header is 24 hour later even though the max-age is just 2,940.

My Worker nor my origin server sets an expires header. This is not a problem because expires is ignored when there’s a cache-control header according to MDN. But I just want to check if there’s no error hidden in my Worker/configuration before I set the worker to site-wide.

#4

@KentonVarda I’m still confused on the precedence of how caching workers in Workers, even though you have stated, it’s evaluated after the worker.

Please see this code - which looks at a URL path, and decides (before this) whether it should be cached or not. If it should be cached, this code runs in our Worker:

 if (isCacheable === true) {
                let cachedResponse = await fetch(request, {
                cf: {
                      // Tell Cloudflare's CDN to always cache 
                      cacheTtl: CACHING_TTL,
                      cacheEverything: true,
                      polish: 'lossy' 
                    },
                })                
                return cachedResponse;
            }

How do I know for sure that a cacheable response will indeed run from cache, and not fetch again from the origin?

i.e. how do I say in the worker - “return cache if it exists for this path”?

#5

Hi @tallyfy,

If you want to make sure the response is only served from cache and never falls back to origin even if there’s no cached response available, then you need to use the Cache API.

#6

Thanks for the note @KentonVarda

I want to check first if there is a response for that URL in cache already, and if there is - serve it immediately.

Otherwise - get the response from origin.

Any easy code/way to do that?

Amit - from Tallyfy

#7

Hmm, that sounds like exactly what a normal fetch() request already does, so I’m not sure I understand what you’re asking.

#8

In that case, my question is answered - since cache is checked first then served if available.

#9

Is there an overview about which features run before and which after workers?

#10

I don’t think there’s a definitive list, but generally, security features run “in front of” workers, and everything else runs “behind”, i.e. on subrequests made with fetch().

We actually want to change this at some point. We’ve realized that features which modify the response, like JavaScript minification, should run on the final response returned from the worker, rather than running on subrequests. But we won’t change it without a careful strategy to avoid breaking already-deployed workers.

3 Likes