Unexpected headers

Hi all,

Trying to write a simple Cloudflare Workers function that makes a request to a certain URL, looks at the header of that request and then returns some results that depend on those headers. I’m doing this with fetch.

When I test the worker in the dashboard, it works as expected and all the headers are exactly what they are when I would make a request to the URL in a browser. However, when I make a request to the worker outside of the Cloudflare dashboard, the headers seem to get mutated in some form?

Some demo code:

addEventListener('fetch', event => {
  event.respondWith(handleRequest(event.request))
})

async function handleRequest(request) {
  const resp = await fetch('https://google.com');
  const headers = [];
  for(const [key, value] of resp.headers.entries()) {
    headers.push([key, value]);
  }
  return new Response(JSON.stringify(headers), { status: 200 });
}

Run this in the CF dashboard and the results are very different to what you get when you call the function manually in a browser.

Edit: here’s a link to a worker with this code https://headers-issue.samjmck.workers.dev, I don’t think a response from Google will have a server: cloudflare header :stuck_out_tongue:

I believe those Cloudflare headers were added automatically in the response, since you are making requests to Cloudflare - it’s just that you use Workers to make another request to Google.

But I’m not really sure about it, let’s see whether other people has some insights about this.

I’m not sure I understand your answer. I expect these headers to be in the response to the Cloudflare Workers request, not in the fetch request that the Cloudflare Worker makes to any URL. It doesn’t really make sense that each fetch request has e.g. the server: cloudflare header, as not every server is using Cloudflare of course.

Does anyone have an idea what this could be?

Hi @samjmck,

This is an expected side-effect of how our system works. When you make an outgoing request from a worker, that request is in fact proxied through Cloudflare in much the same way that a non-Workers
request would be handled by Cloudflare’s CDN. For example, outgoing fetch() requests go through Cloudflare’s HTTP cache, so that you get caching. This is exactly the same HTTP cache that we use for non-Workers sites.

This is one of the known ways in which the dashboard preview does not exactly match production behavior. When you are running a worker in the dashboard preview, the subrequests actually do not go through Cloudflare’s stack – they go directly to the destination server.

We are working on improving the realism of the preview. Note that if you use wrangler dev to run a preview, that preview runs inside the real Cloudflare stack, and so will produce more realistic behavior.

4 Likes

Thanks for the response!

I didn’t know it worked like that. Maybe something to add to the docs?

Is it possible to get the original headers somehow? Because my app relies on those headers and I don’t know how to distinguish the Cloudflare headers from the original headers now.

No, there’s no way to get the verbatim original headers.

How does your app rely on the original headers, exactly? Generally users of the HTTP protocol are expected to ignore headers that they don’t understand.

It’s just a small side project that checks whether a site uses Cloudflare or not. If every site has the server: cloudflare, then the app will think every site uses Cloudflare.

Ah, I see. Yeah I guess it’s unfortunate that the server header gets replaced in this case.

You might have to do something a bit more complicated, like do a DNS lookup (via 1.1.1.1’s API, perhaps) and then check if it returns an address in one of Cloudflare’s IP ranges.

This topic was automatically closed 24 hours after the last reply. New replies are no longer allowed.