Remix on Cloudflare Worker - Using custom domain, fetch request to external API returns 521 status

I’ve setup a Remix site on Cloudflare workers. Cloudflare provides me with a domain name and everything works fine. Now I’ve also setup custom domain name and I can visit all pages on my site except for the page that does a fetch, using loader and useLoaderData to my external API on AWS. In this case I get a status code 521 from this request. Does anyone know what could possibly be causing this error and how to resolve it?