Cloudflare "breaks" auth flow for ASP.NET application

Hi. We run the Cherwell IT Service Management system in which uses ASP.NET. We have this sitting behind Cloudflare mostly for WAF, but caching static assets is a bonus.

About 48 hours ago (Sunday our time - UTC+10) logins to our platform started failing. If we bypass Cloudflare, then they start working again.

Investigations show that the login flow should be as follows:

  1. Client POSTs login form to server
  2. Server responds with 302 redirect to auth api and sets a cookie (.AspNet.WebApiCookie)
  3. Client GETs auth api endpoint and includes the cookie in the request
  4. Server responds with 302 redirect to the application

This is what we are seeing with Cloudflare in the path:

  1. Client POSTs login form to server
  2. Server responds with 302 redirect to auth api and sets a cookie (.AspNet.WebApiCookie)
  3. Client GETs auth api endpoint and DOES NOT include the cookie in the request
  4. Server responds with 302 redirect to the login form (and deletes the cookie)

We didn’t push any changes to the application over the weekend, or make any network or DNS changes.

Any thoughts on this would be much appreciated.

(have raised a ticket as well: 2184156)

Thanks,
Chris Ford

Thanks for the description here - given what you have described - Cloudflare is not stripping the cookie (this would happen if resources are cached by Cloudflare, for example) so in Step 3 there is your client deciding not to transmit the cookie.

So what you’ll need to do here is capture a HAR file with and without Cloudflare and compare the set-cookie header from step 2. You should also check the console log in developer tools of the browser - if the browser is making a security related decision to not transmit a cookie, it should log it in there, too.

There’s nothing “out of the box” that would cause Cloudflare to do this, so this is unusual.

Thanks Simon.

Checked the .har file captures I have again (they’re attached to the ticket), and the set-cookie headers look fine. Also checked the console and nothing gets logged there until we get to the application well past the auth flow.

1 Like

Hi Chris,

So in all of your requests via Cloudflare you will see CF-Cache-Status: DYNAMIC - that means the response headers such as set-cookie (excluding Cloudflare specific cookies) are set by your origin.

Thus, if you see a set-cookie that is breaking your application - or one that is missing - the origin is responsible for making that decision to send or not to send it.

The most common reason I can think of is that you have some IP-based logic on your origin server(s) or load balancers. When you enable Cloudflare, IPs visiting your server will come from our IP ranges and not your real visitors. If your session cookies also expect the visitor IP to be present, you may see misbehaviour behind Cloudflare.

Take a look at how to restore visitor IPs to your application or load balancer here:

Try this and see if it resolves things.

I should add, there is great specific instruction for IIS and ASP.net for restoring visitor IPs here:

1 Like

Thanks Simon, I’ll feed this on to our team.

A question though, what changed? We didn’t push any changes, but something broke over the weekend. Do you know if there were changes on the Cloudflare end?

Hi Chris - no changes here - as I mentioned it’s your origin that is deciding not to send a cookie - we can’t really know why your origin/application decides to do it - the ideas I mentioned above are just good educated guesses, really :slight_smile: .

Hi. Apologies if I was unclear - the origin sends the cookie in both scenarios, it’s the client (browser) side that is not sending the cookie back in the scenario with Cloudflare in the path.

Nothing else is different between the working scenario without Cloudflare, and the non-working scenario with Cloudflare. This has been working for a number of years, it just stopped this weekend, and no one says they changed anything, but someone obviously did.

Hi Chris,

Let me post redacted headers to explain - I think I’ve spotted it for you. Here’s your working response headers from your authorisation flow:

Access-Control-Allow-Credentials: true
Access-Control-Allow-Headers: Content-Type, Accept
Access-Control-Allow-Methods: GET, POST, OPTIONS, PUT, DELETE
...
Date: Tue, 15 Jun 2021 07:09:18 GMT
Expires: -1
...
Set-Cookie: .AspNet.WebApiCookie=REDACTED; path=/; expires=Tue, 15-Jun-2021 07:12:18 GMT; secure; HttpOnly
...

And here’s your non-working (via Cloudflare) response headers:

Access-Control-Allow-Credentials: true
Access-Control-Allow-Headers: Content-Type, Accept
Access-Control-Allow-Methods: GET, POST, OPTIONS, PUT, DELETE
Cache-Control: no-cache
CF-Cache-Status: DYNAMIC
...
Date: Tue, 15 Jun 2021 11:03:35 GMT
...
Server: cloudflare
Set-Cookie: .AspNet.WebApiCookie=REDACTED; path=/; expires=Tue, 15-Jun-2021 11:03:31 GMT; secure; HttpOnly
...

Someone needs to wind the watch on your server - because it is living in the past! It’s issuing a cookie that expires 4 seconds before it was issued - meaning the client won’t store it or send it with subsequent requests.

Fix your server time and I think this should resolve itself.

3 Likes

Simon, you are a legend.

Yes, we did indeed have a time issue - which is pretty embarrassing. Also a bit embarrassing that I did not see that issue with the cookie expiry time myself.

Monitor your server time people and alert when they are out of sync!

Testing looks pretty good, and we’ll be making a prod change tonight to fully confirm.

Interestingly, we have discovered that different browsers behave differently here. With Cloudflare in the path none worked, but with Cloudflare out of the path Chrome and Edge work, but Firefox and Safari do not.

Will update here and mark the Solution once we’ve confirmed in prod.

Thanks for your assistance.

2 Likes

Hi Chris - so pleased it’s working better now - it took me a little while to spot, I must admit.

Just in case you didn’t know - Cloudflare offer an NTP service you can use to keep your machine(s) clocks in sync with our service:

https://developers.cloudflare.com/time-services/ntp/usage

I am not sure why some browsers are behaving differently - my assumption is the browser uses the cookie expiry timestamp and compares to the local system time to understand whether a cookie is expired… but if they used the server Date header, then it might work differently on Cloudflare vs Off - since your origin server’s Date and Cookie expiry timestamp are presumably produced by the same (slow) clock. On Cloudflare, your origin produces the Cookie expiry timestamp using the slow clock and Cloudflare produces the Date header using our clock which is more accurate, thus on Cloudflare the cookie timestamp is older than the current Date header. This is a question for a rainy day though!

1 Like

This topic was automatically closed 3 days after the last reply. New replies are no longer allowed.