Blocking direct access to an API

[farshad]

Hi all

I have the following architecture,

My main website

AAA Northern CA, AK, AZ, MT, NV, UT & WY | American Automobile Assn

API on the same domain

AAA Northern CA, AK, AZ, MT, NV, UT & WY | American Automobile Assn

The API needs authorisation from the user side to receive requests, however I am putting another level of security on it which requires a header field with an encoded value to be present in the request message. So let’s say the header is called X-AUTH .

I need to make sure using Workers I can inject this header field into the request when a call is make to the api endpoint. This only need to happen when the call is to the /api part of the site. Up to this part it’s ok but what is confusing for me is how I can make sure this is only done if the request is coming from within my domain.

So if for example someone tried to use Postman to make a request to the end point AAA Northern CA, AK, AZ, MT, NV, UT & WY | American Automobile Assn it will fail even if the include the header.

So in short, I need to block access to any request that is coming from outside Cloudflare, also I don’t want to use referrers as they are not advised for authentications and some ISPs might remove it from the headers.

Thanks

[farshad]

Sorry forgot to mention this, the Worker rule shouldn’t inject the header field if the endpoint AAA Northern CA, AK, AZ, MT, NV, UT & WY | American Automobile Assn is called on it’s own for example via Postman or any other client. That’s why I need to make sure it is injected a level before that and check who is calling it.

[adaptive]

Maybe this solves your request:

addEventListener('fetch', event => {
  event.respondWith(fetchAndApply(event.request))
})

async function fetchAndApply(request) { 
  if (request.headers.get('x-auth') != '1234567890') {
    return new Response('Sorry, this page is not available.',
        { status: 403, statusText: 'Forbidden' })
  }
  return fetch(request)
}

[farshad]

Thanks for your message, but that’s not what I need.

I am already checking for the header in my code. What I need is a way to inject this header field into the request only if it is originated from my own domain.

[farshad]

Anyone? There must be a way to do this

[shimi]

If what you want is to verify that the request does come from Cloudflare, wouldn’t comparing the connecting IP (“REMOTE_ADDR”) by your API to check if it’s under any of Cloudflare’s subnets: IP Ranges - suffice?

Alternatively you may either make a unique API subdomain and enforce connections at the TLS level to only be accepted if they were made with Cloudflare’s origin pull certificate, or, continue using /api and request the client certificate but make it ‘optional’ in your server config, and if validated, pass the cert data to the app and verify it (more work, because a separate api.aaa.com would be able to make this completely on the web server and never reach the app layer…)

[farshad]

Thanks Shimi

The first option wouldn’t work. The main reason I need this, is that I am using Azure Functions as my API. And because it’s on consumption plan, for each execution I get charged, so am trying to figure out a way so that I block un necessary executions of the API endpoint. So the request shouldn’t even reach the APP or API.
I guess your second suggestion is better for my case.
However the question is that as both domains www.aaa.com and api.aaa.com are from the same domain and both are routed through Cloudflare, would that still work with the certificate? I mean wouldn’t api.aaa.com still think the requests are coming from Cloudflare because itself is a Cloudflare domain?
sorry it got a bit confusing.

Thanks

[shimi]

Why doesn’t it work? Do the requests not come from those IP ranges?

I am not familiar with Azure Functions, sorry.

If you can define two domains in Azure, one www.aaa.com and one api.aaa.com - those are (supposedly) two different entities, even if they point to the same IP. The way HTTP servers differentiate between different domains on the same IP is through the HTTP “Host:” header (and on the SSL/TLS level, through the SNI mechanism).

I did not know about Azure, it wasn’t in your original post. Assumed regular web server, in which you can easily do all these things.

No idea if Azure can even authenticate with a custom client certificate… I try to avoid any service related to Microsoft in any way

[farshad]

No it doesn’t because I can’t rely on IP addresses only, anyone can get a free Cloudflare account and send requests to my API endpoint.
The certificate sounds good. Their function app on consumption plan is almost free, first one million execution per month is free then after that for each million you pay only $0.20, for me it turns out completely free.
Yes it’s a complete web server, that’s why it’s one of the best comparing to the same services on Amazon and Google. I’m exactly opposite I try not to go around Google or Amazon AWS

I think I’m going with the certificate option, thanks a lot for your time and help

[farshad]

I don’t think it is working, it got very confusing and messy. So here is what happened.

I created a new subdomain: api.aaa.com pointed to my API app

I went here and enabled Origin Pulls Authentication

As soon as I did that, I couldn’t access the site anymore it was showing Cloudflare’s page saying that Origin doesn’t have a valid certificate. Good, as this was expected.

Then I downloaded the certificate file from here

https://support.cloudflare.com/hc/en-us/articles/204899617-Authenticated-Origin-Pulls

And installed it here on Azure, as public certificate.

Then the website started working again, meaning Cloudflare managed to see the certificate

However, I am confused here as it is working after I installed the certificate. How am I supposed to authenticate an API request. I mean when I used Postman I still was able to make requests even though the authenticated origin pulls was enabled. Am I supposed to do something else? In workers?

Now apart from what I explained above;

I thought I’m going to delete the certificate which I uploaded earlier, just to see what happens. So I deleted the certificate. Purged the cache on CF, but the website kept working fine even though the Authenticated Origin Pulls was still enabled. How is that possible?

Then I discovered this in the SSL section of my API app on Azure.

I enabled it. Then I disabled the Authenticated Origin Pulls on CF. The website stopped working and it showed this message.

Which is not from CF. It is from Azure.

Now interestingly when I enabled Authenticated Origin Pulls on CF then website started working.

All of these are happening even when I deleted the certificate.

---

Not sure what’s happening but what is very important now is that even though I have a way to block access to the endpoint, the blockage is global so there is no way I can say block access to the public and only allow if the request coming from within www.aaa.com .

What I need is

www.aaa.com ---------------access----------> api.aaa.com
Public Internet ---------------block------------> api.aaa.com

Am I missing anything here?

[cbrandt]

I’m not familiar with Workers (or JS programming), but wouldn’t a Firewall Rule suffice to block these API requests?

(http.host eq “api.aaa.com” and http.referer ne “www.aaa.com”)

then action > block

You mentioned you didn’t want to use referer, but in this case, since you’d be using the ‘not equal’ operator, as long as your own server provides the referer, you shouldn’t worry about this header being stripped by ISPs.

[farshad]

Interesting, I’m very much new to CF, don’t know much about their firewall, I will try that. It seems to be a solution.
Thanks a lot

[farshad]

I added this to the firewall it seems to be working but the problem is that my API is accessed via JavaScript, so the browser and ISP of the client is involved, that means the referrer header can be stripped by either browsers or ISPs.

[Judge]

@farshad

stripped by … or ISPs

Is all of this traffic over HTTPS? ISPs should not be able to change/read/modify any data of a request if the request goes over HTTPS since they can’t decrypt or change the encrypted message.

---

As for trying to block connections via the API, you’re fighting an uphill battle. Just requiring the Referrer header is a good start for what you’re trying to do, but it won’t stop someone dedicated enough. If a browser has access to an API, that means anyone with enough proficiency could just look at the chrome/FF devtools network log and see all of the headers, request parameters, etc. and simply copy those headers into a script that abuses this open access API.

The only you can really do is raise the bar to entry. Maybe encode/encrypt the request in the browser then decode/decrypt the request when it gets to your server. Maybe require some sort of proof of work on the browser so that someone looking to abuse it wastes their CPU power solving hashes, or (at last resort) require ReCaptcha to verify that requests come from a human.

Or, for the best solution by far, you could make your API require authentication. With an API that requires login/auth, you can then tie operations to a user and run some basic abuse analysis that makes sure they’re not hitting the API 10 times a second or anything.

[farshad]

Thanks Judge

Yes all the request are over HTTPS, I have forced that for the entire application.

Sorry I think I should have explained it in details. All the endpoints and APIs are protected except one which allows people look up a specific content type. For that one I have created a logic to detect abusive IP addresses and add them to CF firewall but need to make sure no automatic or unnecessary request is made to the endpoint because my provider which is Azure will be charging me for each execution, this could be happening even by bots. So I need to not only block bots but also anyone who tries to access the endpoint by just copying the URL of the API and try to mess with it.

The application is a Single Page Application using VueJS so the whole client is running on the browser side and is JavaScript. And yes it’s possible that they can view the network traffic, but with https this is going to be very difficult for them to see any details passed to the API.