Anyone know how do i detect GTmetrix agent on workers?

Developers Workers
1

I am trying to detect GTmetrix agent but cant get it working on workers. Here is the code how i am trying. Thanks!

const html = `

Hello World

This markup was generated by a Cloudflare Worker.

`;

async function handleRequest(request) {
let userAgent = request.headers.get(‘User-Agent’) || ‘’
if (userAgent.includes(“GTmetrix”)) {

return new Response(html, {
headers: {
“content-type”: “text/html;charset=UTF-8”
}
})
}

else {

return fetch(request)
}

}

addEventListener(“fetch”, event => {

return event.respondWith(handleRequest(event.request));
})

2

From what I’m seeing it does not appear that GTmetrix always has GTmetrix in it’s user-agent.
Dumping the headers shows that the user-agent did not include GTmetrix when I had pointed it at my Worker.

image
image1911×1241 298 KB

As far I’m aware there’s no other way to detect whether GTmetrix is running on your site

3

On firewall rules it works it shows the expression like this

(http.user_agent contains “GTmetrix”)

Not sure how to do implement on worker

4

Just tested against my own website and logged the requestheader.

Looks like this:

Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/86.0.4240.193 Safari/537.36 GTmetrix

So actually I can see the trailing GTmetrix and you could identify it by this string.

5

That’s odd, because I’m not getting that user-agent on my Worker and also not when using a RequestBin

image
image2338×927 168 KB

1 Like
6

I am trying to detect by “GTmetrix” but it seems not working on worker, on firewall it works

(http.user_agent contains “GTmetrix”)

7

I have not tested it via workers apparently.

I have tested it against a normal URL on my Server.

Logged in at GTmetrix:
Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/86.0.4240.193 Safari/537.36 GTmetrix

Anonym GTmetrix:
Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/86.0.4240.193 Safari/537.36 GTmetrix

It definitely shows up with the “Gtmetrix” in my logs (serverside). But I actually dont know how to detect it via Workers, or why it does not show up there. Maybe it gets cut off there?

But you’re right. When I check what GTmetrix shows me as “request Header” it is not part of the user-agent:

GTmetriy request header
GTmetriy request header727×24 2.14 KB

On my server it shows up anyway as I catch the raw header and save it to a DB

As the syntax is:
User-Agent: <product> / <product-version> <comment>
(Source: HTTP headers | User-Agent - GeeksforGeeks)

Maybe the comment is “GTmetrix” and is getting cut of by Cloudflare? I cant tell sorry. Maybe the OP should create a ticket to ask CloudFlares support if there is a way to detect/fetch the raw full user-agent?

But the approach of the OP seems to be right. As this example uses the same method:

  const userAgent = request.headers.get("User-Agent") || ""
  if (userAgent.includes("bot")) {
    return new Response("Block User Agent containing bot", { status: 403 })
  }
1 Like
8

Hi,

I had a use case to detect certain bots and serve response from cache instead of making a call to origin.

We wrote the following method to use a regex pattern to determine the agent. Better to make the regext pattern configurable than hardcode it in the code.

export async function checkIfBot(request) {

//TODO: get from BrowsePagesCommonKV
let botPattern = “(googlebot/|Googlebot-Mobile|Googlebot-Image|Google favicon|bingbot|AdsBot-Google-Mobile|APIs-Google|AdsBot-Google|Googlebot-News|Googlebot-Video|Mediapartners-Google|googleweblight)”

const patternConfig = await getCache(“BOT_PATTERN_REGEX”, BrowsePagesCommonKV)
if (patternConfig) {
botPattern = patternConfig
}

const re = new RegExp(botPattern, 'i')
const userAgent = request.headers.get('User-Agent')

let url = new URL(request.url)
console.log(url.href)
let isBotRequest = false
if (re.test(userAgent) || url.href.includes(“cache-test”)) {
console.log(‘the user agent is a crawler!’)
isBotRequest = true
}

return isBotRequest

}