Overriding CF-Connecting-IP in subrequests destined for a non-Cloudflare customer zone

[bayo]

It seems that it’s possible to override the CF-Connecting-IP and X-Forwarded-For headers with arbitrary values for subrequests destined for non-Cloudflare zones via the X-Real-IP header. Here is a minimal POC:

addEventListener("fetch", (event) => {
  event.respondWith(
    handleRequest(event.request).catch(
      (err) => new Response(err.stack, { status: 500 })
    )
  );
});

async function handleRequest(request) {
  const headers = new Headers(request.headers)
  headers.append("X-Real-IP", "8.8.8.8")

  const newRequestInit = {
      method: "GET",
      redirect: "follow",
      headers: headers,
    };
  const attrs = new Request(request, newRequestInit);
  const newRequest = new Request(
      "https://example.com",
       attrs
  );
  return fetch(newRequest);
}

This is the resulting HTTP request that’s made:

GET / HTTP/1.1
host: example.com
connection: Keep-Alive
accept-encoding: gzip
x-forwarded-for: 8.8.8.8
cf-ray: 6d743456d21ad45f-HAM
x-forwarded-proto: https
cf-visitor: {"scheme":"https"}
cf-ew-via: 15
cdn-loop: cloudflare; subreqs=1
accept-language: en
accept: */*
user-agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/97.0.4692.71 Safari/537.36
cf-connecting-ip: 8.8.8.8
cf-worker: <redacted>

Given that it’s not possible to achieve this by modifying the CF-Connecting-IP or X-Forwarded-For headers directly, I’m wondering if this is intended behaviour. Moreover, the relevant documentation says:

For Worker subrequests destined for a non-Cloudflare customer zone, the CF-Connecting-IP and x-real-ip headers will both reflect the client’s IP address, with only the x-real-ip header able to be altered.

I might be misunderstanding this sentence, but I’d expect CF-Connecting-IP to always contain the client’s IP address, even if the X-Real-IP header is altered.

Interestingly, this answer from 2019 implies that the X-Real-IP header is renamed to CF-Connecting-IP which I assume would explain this behaviour (if it’s still actual) and also imply that this is indeed not expected behaviour (given that the answer indicates it’s a bug).

This might have security implications for anyone who assumes that the CF-Connecting-IP header can be trusted if it’s coming from Cloudflare’s IP ranges (admittedly, given that the subrequest is destined to a non-Cloudflare zone, this assumption might simply be wrong). I wasn’t able to find any other relevant documentation about that fact though.

Is it expected that CF Workers can set arbitrary CF-Connecting-IP and X-Forwarded-For headers in subrequests destined to non-Cloudflare zones?

[KentonVarda]

When making a request to a third-party non-Cloudflare zone, it is intended that you can override the header to whatever you want. Such a third party is not a Cloudflare customer and so has no reason to think that Cloudflare is going to intercept these headers and authenticate them as trustworthy. If the third party actually is a Cloudflare customer and is using IP allowlisting to restrict requests to only come from Cloudflare, they must also verify that Cloudflare actually processed the request on behalf of their zone. One way to verify this is by validating the Host header: If the Host header matches a Cloudflare zone, then Cloudflare has processed the request on behalf of that zone and applied its security settings. If the Host header does not match any Cloudflare zone, then Cloudflare has assumed the host is not a Cloudflare customer and has not applied any security protections on their behalf. These “security protections” include validating the client IP address.

(Other, better ways to verify Cloudflare was actually acting on behalf of your zone include using Authenticated Origin Pulls with per-zone certificates, or using Cloudflare Tunnel . If you don’t use any of these mechanisms and instead rely only on the IP address being a Cloudflare IP, then an attacker can easily bypass your Cloudflare settings by creating their on Cloudflare zone and configuring their DNS to point to your origin server. Requests sent to the attacker’s zone will then apply the attacker’s security settings – which could all be disabled – and then would be sent to the victim’s origin, from a Cloudflare IP. Notice this attack does not require Workers.)

Getting back to your original question, there is a known non-security bug here: The X-Real-IP header is a quirk of our system that we never really intended to expose. Instead, you should be able to set the CF-Connecting-IP header directly. But, in practice it’s always been the case that CF-Connecting-IP is copied from X-Real-IP, so it turns out if you do want to override it, you have to do so by overriding X-Real-IP. That wasn’t something we consciously designed that way, it was sort of an accident. What I recommend you do is override both headers (X-Real-IP and CF-Connecting-IP), so that if we ever “fix” this behavior, your code will still work with the fix.

[bayo]

Thanks for the detailed answer!

If I understand correctly, this means that third-parties (non-Cloudflare customers) must not trust the CF-Connecting-IP and X-Forwarded-For headers, even when the requests originate from Cloudflare IPs. That sounds fair, but I’m worried that a lot of service providers do make this erroneous assumption.

If you don’t use any of these mechanisms and instead rely only on the IP address being a Cloudflare IP, then an attacker can easily bypass your Cloudflare settings by creating their on Cloudflare zone and configuring their DNS to point to your origin server. Requests sent to the attacker’s zone will then apply the attacker’s security settings – which could all be disabled – and then would be sent to the victim’s origin, from a Cloudflare IP. Notice this attack does not require Workers.)

This makes complete sense regarding Cloudflare security settings and for Cloudflare customers. However, unless I’m missing something, I think it’s not entirely relevant for the discussed topic, because without the use of Workers, I don’t see any way how creating a Cloudflare zone allows to spoof the CF-Connecting-IP and X-Forwarded-For headers in requests from Cloudflare IPs that are destined to the victim origin. That is since a “HTTP Request Header Modification” transform rule will not let you override the CF-Connecting-IP, X-Forwarded-For, or X-Real-IP headers.

[KentonVarda]

It may be the case that certain Cloudflare security features are normally not possible to turn off. Nevertheless, it would be inappropriate for anyone to be trusting those features just because a request came from Cloudflare IP, without also authenticating that the request was processed on behalf of their zone.

That said, one thing you can trust is that if the request came from a Worker, it will always have the CF-Worker header indicating the domain name that owns the Worker. This cannot be forged nor disabled under any circumstances.