Fetch.post with basic auth fails on edge (but works in preview)


#1

We want to post to our backend from the edge using basic auth and custom port numbers. It works just fine in the preview, but we get 401 Unauthorised when running in the edge environment.

We did a minimal test with the code below to make sure that it’s a difference between the preview and the edge:

async function log() {
const init = {
method: ‘POST’,
headers: {
‘content-type’: ‘application/json’,
authorization: ‘basic YRNhc3Q6T0JpUkVmZVJiYQ==’,
},
body: JSON.stringify({
message: ‘test’
}),
};

return fetch(‘http://example.com:1234’, init)
}

async function fetchAndApply(request) {
try {
await log();

return new Response('Hello');

} catch (err) {
return new Response(err.message);
}
}

addEventListener(‘fetch’, async (event) => {
console.log(‘Hello’);
event.respondWith(fetchAndApply(event.request));
});

Any idea why it’s only working in the preview?

Thanks,
Markus


#2

Think I figured out what’s happening… When I run code in the preview it makes a POST to the backend as expected. But when it runs in production it makes a GET?? Is this a bug in the fetch implementation or am I doing something wrong in the sample code above?


#3

Maybe @KentonVarda can check this out.


#4

Thanks!


#5

Is it possible that your backend is returning a 301, 302, or 303 redirect? If so, the runtime will, per the Fetch spec, follow the redirect with a GET, not a POST.

To explain the difference in behavior between the playground and production, perhaps the playground is making https requests unconditionally, even when the URL is an http URL (this would presumably be a bug, if so), while the production environment is correctly making an http request, then getting redirected to https?

You might be able to debug further by adding redirect: "manual" to your init blob, then serializing the response you get. Something like this:

async function log() {
  const init = {
    method: ‘POST’,
    headers: {
      ‘content-type’: ‘application/json’,
      authorization: ‘basic YRNhc3Q6T0JpUkVmZVJiYQ==’,
    },
    body: JSON.stringify({
      message: ‘test’
    }),
    redirect: "manual",  // <-- REDIRECT MODE ADDED
  };

  let response = await fetch(‘http://example.com:1234’, init)
  // SERIALIZE RESPONSE
  return new Response(
    `${response.status} ${response.statusText} Location: ${response.headers.get("Location")`)
}

async function fetchAndApply(request) {
  try {
    return log();
  } catch (err) {
    return new Response(err.message);
  }
}

addEventListener(‘fetch’, async (event) => {
  event.respondWith(fetchAndApply(event.request));
});

Edit: changed redirect mode to "manual" in the code snippet. I originally accidentally wrote "follow", which is the default.


#6

I noticed the difference in behaviour using a local node service with ngrok from the edge worker, so I’m pretty certain that it’s not any redirects involved.
Also verified with httpie that our log service sends a 200 back without any redirects:

http POST "http://user:[email protected]:1234" <<<'{ "message": "test }'
HTTP/1.1 200 OK
Content-Length: 2
Content-Type: text/plain

ok

Are you able to recreate this on your end?


#7

Turns out that the request was upgraded from http to https which caused a redirect. We’ll try to skip the upgrade and hopefully it will work then.


#8

The plot thickens… It seems that the 301 redirect isn’t from our servers.

If you try out the following script that posts to requestbin you’ll see that there’s something funny going on:

async function log() {
  const init = {
    method: 'POST',
    headers: {
      'content-type': 'application/json',
    },
    body: JSON.stringify({
      message: 'test'
    }),
    redirect: "manual",  // <-- REDIRECT MODE ADDED
  };

  let response = await fetch('http://requestbin.fullcontact.com/ve6frrve', init)
  // SERIALIZE RESPONSE
  const r = `${response.status} ${response.statusText} ${response.headers.get("Location")}`;

  return new Response(r);
}

async function fetchAndApply(request) {
  try {
    return log();
  } catch (err) {
    return new Response(err.message);
  }
}

addEventListener('fetch', async (event) => {
  console.log('Start');
  event.respondWith(fetchAndApply(event.request));
});

Use this url to fetch the results: https://requestbin.fullcontact.com/ve6frrve?inspect


#9

It turns out that the production environment for workers doesn’t seem to read the port part of the url’s. We setup a new logger on port 80 and now it works fine. Is this the intended behaviour?


#10

Hi @dev35! I apologize for my delay in responding.

The intent is for ports in URLs to be honored, so it sounds like there might be a bug here. I’m confused that your new logger on port 80 is receiving requests, since previously you were seeing a 301 to https (presumably generated by Cloudflare, before the origin). Is that no longer the case?

I suspect that your SSL setting in the Cloudflare dashboard (under Crypto -> SSL) may also be coming into play here. Could you try changing your SSL setting and seeing if that influences the behavior you’re seeing?

Harris


#11

We were seeing redirects before as it wasn’t hitting the port we expected. The domain we used before automatically upgrade to ssl on port 80, but we specified another port in the fetch request.

When we changed to use a separate subdomain for the logger (so we could use port 80) it worked as expected.

We disabled http->https redirects for the cdn, but I don’t think that made any difference.


#12

Hi @dev35, I’ve been a bit too busy to investigate this deeply, but I wanted to let you know that I haven’t forgotten about this. I’ll let you know when I have a chance to revisit.

Harris


#13

Hi,

Chiming in. It looks like the custom port is the problem.

I’m attempting to fetch from https://<mydomain>:7473 and getting:

{
    "webSocket":null,
    "url":"https://...:7473/myurl",
    "redirected":false,
    "ok":false,
    "headers":{},
    "statusText":"Origin Connection Time-out",
    "status":522,
    "bodyUsed":false,
    "body":{"locked":false}
}

The JavaScript is:

const response = await fetch(API_URL, {
  method: 'POST',
  headers: new Headers({
    Accept: 'application/json;charset=UTF-8',
    Authorization: 'Basic TOKEN',
    'Content-Type': 'application/json',
  }),
  body: JSON.stringify({...}),
})

I put a temporary NGINX reverse proxy in front of the service, and it worked.

Notes

  • The worker always works in the Testing tab.
  • That the URL I am hitting is also a Cloudflare domain. I can’t turn on the DNS and traffic to this subdomain either, because the ports aren’t supported. 7474 (HTTP), 7473 (HTTPS), and 7687 (BOLT also secure).