Facebook now adds fbclid query string to URLs, busting CloudFlare's cache

#29

@beeroslav, Page Rules work perfectly fine for me on mobile.

Maybe you have a setting that’s blocking them?

Just a thought.

1 Like

#30

You can’t do anything about this issue from server side, also 302 redirect for each request is really bad practice and it will increase page load time anyway.

If performance is your main goal and you are willing to pay extra $5 per month, this can be easily solved using Cloudflare Workers. You can remove fbclid using Worker, fetch original url without fbclid and it will be cached on the edge. Worker will serve additional requests to this url from cache instead of hitting your server despite fbclid value.

Above mentioned $5 will cover 10 million requests to your url every month and if you have more than 10 million requests monthly, you have to pay extra $0.5 for per million additional requests.

If OP and others are interested in this approach, tommorow I can share tiny code for Workers to solve this issue.

3 Likes

#31

Thanks @nikoloz! I already shared a little piece of code for workers:

addEventListener('fetch', event => {
  let url = new URL(event.request.url)

  if (url.searchParams.has('fbclid'))
   url.searchParams.delete('fbclid')

  event.respondWith(
    fetch(url, event.request)
  )
})

I think we’d all appreciate if you shared your code as well though. The more the merrier!

1 Like

#32

Oh, I’m on mobile and somehow missed it. Your code looks legit, I was going to share something similar as well :+1:

Although, I might be able to reduce this cost drastically. Let me test some things tomorrow and I will report back :slight_smile:

Can you tell us what percentage of this 150M visitors are referred from Facebook?

1 Like

#33

Those aren’t visitors, but rather all requests made to CloudFlare for my site, including all static files (images, css, js, etc.) Regardless, I can tell you that close to 90% of my traffic comes from Facebook.

Can’t wait to hear about your findings!

In the meantime, here’s the Worker script & routes I currently use:

Worker script:

addEventListener('fetch', event => {
  let url = new URL(event.request.url)

  if (url.searchParams.has('fbclid'))
   url.searchParams.delete('fbclid')

  event.respondWith(
    fetch(url, event.request)
  )
})

Worker routes:

On: https://www.website.com/*
Off: https://www.website.com/wp-content/*

Theoretically, these routes ignore static files and only filter Worker requests with permalinks & fbclid parameter. Though adding the Off parameter did indeed drastically lower the amount of requests, it seems there are still more requests coming in than page views. So I assume something might not be quite right yet.

2 Likes

#34

nice @janvitos

curious if you have tested the page speed metric/load time difference between

  1. doing a 301/302 redirect at cloudflare level vs
  2. 301/302 redirect via nginx origin vs
  3. cf worker script to remove fbclid ?

you did mention a 301/302 redirect added ~80ms for you so was curious what each of these workarounds did latency wise ?

1 Like

#35

Indeed I tested the CloudFlare 302 and it adds around 80 ms to the request. The worker should add almost nothing (<1ms), as stated in CloudFlare’s documentation (https://developers.cloudflare.com/workers/faq/):

How fast are Workers?

Even though they’re written in JavaScript, your Workers ultimately execute as dynamically compiled machine code. Most simple Workers can be expected to respond within a millisecond, adding no measurable delay to your requests and responses.

Though, I did not test the 302 at the server level as it’s not an option that’s appealing to me at the moment. I rather do everything at the CloudFlare level since requests benefit from the CDN.

By the way, I’m using the worker now and I seem to be getting under the 10 000 000 requests per month limit (or ~ 333 333 requests per day).

3 Likes

#36

I try few things but looks like your approach with Workers is only available option right now.

Worker routes:

On: https://www.website.com/*
Off: https://www.website.com/wp-content/*

This is smart move as well to reduce Worker requests :+1:

:+1:

1 Like

#37

You could probably specify the route to match the query string reducing the invocations to only the ones that actually need it. Didn’t have a chance to try it, don’t know if it can be done!

2 Likes

#38

Although you can match query string in page rules, sadly it’s not supported in Worker routes. If you try, you will get following error:

Route pattern should not have query parameters

Also there is limitation on wildcards which will make matching query parameters pointless even if they were allowed:

Route pattern may only contain wildcards at the beginning of the hostname and the end of the path

@KentonVarda @zack is it possible to remove this limitations in future?

4 Likes

#39

By the way, I’m actually quite startled that CloudFlare didn’t participate in this thread yet. I’d be really interested in hearing your view on this CloudFlare, and I think I speak for everyone here. Thanks!

1 Like

#40

But if someone use this method

http://www.example.com/?fbclid= forwarding 302 to http://www.example.com/$1

We might not be able to track user in GA. Plus Facebook might take offense they all topic getting redirected.

1 Like

#42

I agree Facebook might not like this, but as mentioned preiously by @boynet2, many websites use URL shorteners on Facebook (DailyMail for example) and don’t seem to be penalized. And URL shorteners are essentially a 301 / 302 redirect that point to your site.

2 Likes

#43

If the FB query string is the first query, would it have the ampersand in front of it? Does FB append the string to existing query strings? In that case should it come last in the regex?

0 Likes

#44

Those of us on the free tier may have already used up our page rules, hence the need to use .htaccess or vhosts config files.

0 Likes

#45

if your urls has parameters then yes you will need 2 page rules

0 Likes

#46

This seems to work for Apache. You can put it in your .htaccess file. You can test it here. Be very careful about where you place it in the file as you can end up in permanent redirect hell, especially if your browser caches redirects! As always, it’s good to enclose this code in <ifModule mod_rewrite.c></ifModule> tags.

The first rule is for URLs that have only the FB query string. The second is for URLs with the FB string and additional query strings. Test the 2nd by adding &?foo to your URL.

   RewriteEngine on
   RewriteBase /
   RewriteCond %{REQUEST_FILENAME} -f
   RewriteCond %{QUERY_STRING} ^(.*)?fbclid=[^&]+((?!&?).)*$ [NC]
   RewriteRule ^/?(.*)$ /$1? [R=302,NE]
   RewriteCond %{QUERY_STRING} ^(.*)?fbclid=[^&]+&?(.*)$ [NC]
   RewriteRule ^(.*)$ /$1%1%2 [R=302,L,NE]
1 Like

#48

Here what you can do to remove Facebook fbclid from args using Nginx:

# remove GET parameters:
if ($args ~* (.*)fbclid=[^&]*(.*)) {
set $args $1$2;
set $removearg "removearg";
}
# cleanup any repeated & introduced:
if ($args ~ (.*)&&+(.*)) {
set $args $1&$2;
}
# cleanup leading &
if ($args ~ ^&(.*)) {
set $args $1;
}
# cleanup ending &
if ($args ~ (.*)&$) {
set $args $1;
}
if ( $removearg = "removearg" ) {
rewrite ^(.*)$ $uri permanent;
}
1 Like

#49

Appreciate the code. Where does it go? Within each ‘server {…}’ or outside of the services?

Right now I have this outside of the services:

map $request_uri request_uri_path { "~^(?P<path>[^?]*)(\?fbclid.*)?" $path;
}

Should it be removed?

Thanks

0 Likes

#50

Yes remove the map configuration and add my configuration inside each server { } block.

1 Like