Allow direct ip access from workers and all headers with fetch


#1

I wrote up a load balancing script that queries an api to get a list of ip address of healthy vms, and everything worked great in the editor. However, when I saved it and tested it live, I would always get an error that says “Direct IP access not allowed”.

I got around that by using a reverse ip hostname (i.e. x.x.x.x.bc.googleusercontent.com). But then, I was also using arbitrary port numbers for my services, and it would just give me an error saying that it failed to connect to the server. This also reveals my hostname, which, ideally, would be kept private.

So, I changed the all the ports to 80. The requests are finally getting through, but it seemed like not all of my headers were being sent. (i.e. Host and X-Forwarded-Proto) I double checked this with httpbin.org/headers.

All of these things make it seem like workers aren’t as powerful as it’s been made out to be. Is it really necessary to make fetch so restrictive?

Additionally, it would be very helpful if the editor testing would succumb to the same restrictions as it does live. Documentation about them would be great too.


#2

Hi @ssttevee,

I’m sorry to hear that you’ve run into so many of our sharp edges – I know how frustrating it can be, and I really appreciate the time you spent writing up your experience, because it helps us learn and improve our service. We definitely have some limitations, and we’d like to lift them as we are able to.

At this time we do not allow direct IP access via fetch(). Although it is technically possible for us to lift this restriction, we must perform a thorough security review of the implications on our infrastructure before we can do so. This will take time, so I cannot guarantee that it will happen by any particular date.

The reverse IP hostname is a nice trick! Regarding the error revealing your hostname, it may be necessary to sanitize certain 5xx errors which Cloudflare returns before returning them to the eyeball. This would probably be necessary even if direct IP access worked, since an origin connection error could still occur in that case, potentially revealing the origin IP.

I think that not being able to use arbitrary ports on an external service is a bug, but I’m still trying to confirm – there may be some rationale for disallowing it that I’m unaware of.

Neither of the two headers you listed can be directly controlled – the Workers runtime will overwrite both of them, since it emits host-style HTTP requests, which it must construct from the absolute URL passed to fetch(). However, both headers should indeed be sent. httpbin.org/headers does not seem to expose X-Forwarded-Proto, but it does expose the Host header, at least when I tested it.

To verify that X-Forwarded-Proto is sent (and correctly reflects the eyeball <-> Cloudflare connection protocol Edit: @ssttevee points out in the next post that it is actually derived from the scheme of the URL passed to fetch()), I configured nginx on my origin with an access log like so:

log_format access_log_format '"$http_host" "$http_x_forwarded_proto"';
server {
  # ...
  access_log /var/log/nginx/nginx.vhost.access.log access_log_format;
  # ...
}

I then deployed the following pass-through worker on my zone. Note that it strips headers that it receives, and does not add any particular headers.

addEventListener('fetch', event => event.respondWith(fetch(event.request.url)))

When I curl http://my-domain.com, I see an access log of the form "my-domain.com" "http" logged at my origin. When I curl https://my-domain.com, I see "my-domain.com" "https" logged at my origin. Are you able to perform a similar test?

I presume the default value of X-Forwarded-Proto (reflecting the connection protocol between the eyeball and Cloudflare) should suffice for your use case, but I understand the need to override the Host header. The best I can offer here might be to point you to the Resolve Override feature:

However, Resolve Override can only resolve to orange-cloud records on your zone, so using this would require you to make an A record for each IP you want to load balance across, which might not be feasible.

I hear you, and we’re working on both making the preview service more accurate and documenting all of these restrictions.

Harris


#3

Hi @harris,

Thanks for the response.

Are the status codes, that I should worry about, in this KB article exhaustive?

I just check this again with port 12345 and 8080. I get a 521 Origin Down.

I tested with this simple go program and with this worker script:

addEventListener('fetch', event => event.respondWith(fetch("http://x.x.x.x.bc.googleusercontent.com/")))

The host header is consumed by the net/http package but the X-Forwarded-Proto is just wrong. Same result for http or https.

Cf-Visitor: {"scheme":"http"}
Cf-Ew-Via: 15
Cf-Connecting-Ip: 2a06:98c0:3600:0:0:0:0:103 // edge node ip?
Connection: Keep-Alive
Accept-Encoding: gzip
Cf-Ray: 48559b3d2382bb5a-SEA
X-Forwarded-Proto: http

I tried passing the request headers in the worker script:

addEventListener('fetch', event => event.respondWith(fetch("http://x.x.x.x.bc.googleusercontent.com/", {
    headers: event.request.headers,
})

I get info about my browser now

Cf-Ray: 48559dea32cdbb5a-SEA
Cf-Ew-Via: 15
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/70.0.3538.110 Safari/537.36
X-Forwarded-Proto: http
Cf-Visitor: {"scheme":"http"}
Accept-Encoding: gzip
X-Forwarded-For: [redacted] // my ip address
Cache-Control: max-age=0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8
Cf-Connecting-Ip: [redacted] // my ip address
Cookie: __cfduid=dca2d53b60560ebc444310f78292b5ce91544170678
Upgrade-Insecure-Requests: 1
Connection: Keep-Alive
Accept-Language: en-US,en;q=0.9

It looks like it might be taking the protocol from the url that I pass to fetch. I got around this issue by just passing the same information through non-standard headers like X-Host.

I also tried reading the request headers directly from the worker to see the difference:

accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8
accept-encoding: gzip
accept-language: en-US,en;q=0.9
cache-control: max-age=0
cf-connecting-ip: [redacted] // my ip address
cf-ipcountry: CA
cf-ray: 4855b854ea12bb12
cf-visitor: {"scheme":"https"}
connection: Keep-Alive
cookie: __cfduid=dca2d53b60560ebc444310f78292b5ce91544170678
host: [redacted] // my website
upgrade-insecure-requests: 1
user-agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/70.0.3538.110 Safari/537.36
x-forwarded-proto: https
x-real-ip: [redacted] // my ip address

Interestingly, x-real-ip and cf-ipcountry are also excluded…


#4

Hi @ssttevee,

No. I know that we return 500, 502, 504, and 520-526, inclusive, but I’m unaware of an exhaustive list in our documentation. If you need to sanitize the error pages, I’d recommend unconditionally replacing the bodies of 5xx responses with your own text.

Thanks for pointing this out; I was mistaken: X-Forwarded-Proto takes its value from the URL passed to fetch() (except same-zone subrequests with a Flexible SSL setting, which always take the value “http”, it seems).

Harris