Cloudflare adds ~200ms to all requests and sometimes 5+ seconds

So I been having big troubles with our website lately where all pageloads have been significantly slower and some page loads is really slow 5-30 sec. This is with a fairly small amout of traffic (request throughput of under 1 req/s)

The web is on a heroku premium dyno and is node based.
The server is on a another heroku premium dyno and is written in Go.

The browsers/users is mostly located in Sweden.

I did som testing myself:
The site is located on (pointing to heroku): www.borskollen.se
The api is directly reachable (bypassing CF) on: https://bk-eu.herokuapp.com

Issue #1
When caching in CF is disabled, put in development mode at CF, the extra time it takes by going through CF is almost 500ms on a request that usually takes around 50ms… This happens on all requests and on pages where we do 3-7 request this is devestating on the total load time on our web… Even if cache is enabled on CF the extra amount of time is around 200ms for every request…

An example of 100 reqs:

https://bk-eu.herokuapp.com/api/featured/links ~ 30 ms, p100 ~45ms
https://www.borskollen.se/api/featured/links ~300ms, p100 ~560ms

Issue #2
The extra latency on all request is one issue. The second and more annoying one is that when using CF (borskollen.se) it sometimes takes 5-30 seconds before getting an answer! I’ve verified that the server took just a few ms to process the request and the issue happens along the way. This does not happen when skipping cloudflare.

Here is an illustration of the web dyno when I changed the endpoint for all requests from borskollen.se to bk-eu.herokuapp.com Here it’s also fairly clear with the spikes in load time we get once in a while…

Some load time numbers on our web in general from the graph above:

Using borskollen.se/api as endpoint for api calls (Cloudflare):
Max ~ 14.9s
99p ~ 5.3s
95p ~ 1.1s
50p ~ 0.5s

Using bk-eu.herokuapp.com/api as endpoint for api calls (not Cloudflare):
Max ~ 1.3s
99p ~ 0.3s
95p ~ 0.2s
50p ~ 0.06s

As you can see there is an substantial difference

As I am aware I have not really had this issue before and my feeling is that this appeared a couple of weeks ago for the first time.

How can I approach this? I would prefer to use our borskollen.se domain for all requests since we have apps as well and I don’t want to hard code the heroku address if we switch underlaying infrastructure one day…

Any ideas here how to proceed with this?

Here is some more debugging info for you at Cloudflare.
Tested 30 requests after each other (borskollen.se) and 2 of 30 came out 1sec+. The rest were ranging between 120-400ms. A request that takes ~90-100ms directly to my server (bk-eu.herokuapp.com).

2 better ones

cf-request-id: 07f6aace3e000016a1290f8000000001 ~ 122ms
cf-request-id: 07f6ad898e000016a1381e5000000001 ~165ms

2 really bad ones

cf-request-id: 07f6a51d86000016a1212c4000000001 ~13.2 sec
cf-request-id: 07f6aa2544000016a18f042000000001 ~1.82 sec

If you haven’t already, please make a Cloudflare support ticket with all the debug information you have listed, and post the ticket number in this thread

1 Like

Hi,

Created a support ticket yesterday actually. The ticketnumber is: #2076238

1 Like