Worker identified as a bot, any ideas?

I have an issue where I am doing a fetch in a worker, but the service I’m connecting to seems to be blocking the CF worker IP (and GCP, and AWS) as being a bot. Does anyone have any clever ways around this?

When I run with ‘wrangler dev --local’ all is well, but as soon as I run non-locally or publish, the fetch is blocked by the end service.

In an ideal world I could identify with the origin as the client IP (I know that’s not an option), so I’m a bit out of ideas.

Thanks!

There isn’t any way around it really - cross-zone requests use a special IP address since your Worker might be making requests on the behalf of the user without their knowledge, possibly maliciously, so the IP isn’t sent so it can’t be linked to them.

Even if that wasn’t the case, the CF-Worker header is always sent so they could just as easily block any requests with that request header.

You could make a proxy for your requests but that’d be outside the Worker scope.

4 Likes

Yeah, tricky one really. I tried adding a proxy via AWS and that’s blocked too! The easiest solution right now seems like using Miniflare to run my code from a non blacklisted server (e.g. Oracle Cloud), but that’s just insane! :grinning:

If you have a contact for the site owner, you can advise that they move away from blocking the IP and instead apply blocking/rate-limiting based on the CF-Worker header.

The IP in question is used by every Cloudflare Worker everywhere for any cross-zone requests - that’s to say, requesting anything that isn’t the same domain/account it’s on.

The CF-Worker header contains the domain that the Worker is from, i.e CF-Worker: example.com.

Of course, this requires people to especially handle Workers and also can’t be done at the same layer as just blocking an IP/port - headers only exist in HTTP at Layer 7 as opposed to the TCP/IP stack at Layer 3. It’s unlikely, but if you’re a customer of the service then you might have some more sway.

Cloudflare Workers are intentionally identifying themselves as such for both site owners to be able to block abuse and also to protect users of your Workers. If it wasn’t and I visited your Worker, you could go execute malicious subrequests like SQL injection against a site and it’d be linked to my IP address - that’s obviously something that wants to be avoided.

2 Likes

Unfortunately that’s not an option!

Maybe one day Cloudflare could offer BYOIP CF Workers :smiley:

1 Like

This topic was automatically closed 3 days after the last reply. New replies are no longer allowed.