Slow Website if routed through CF

[M4rt1n]

I do not use CloudFlare “Cache Everything” because of dynamic content.
But I noticed at my Website which uses “FRA” as CloudFlare-Ray that my TTFB is very very high.

When I call a site which is getting routed through CloudFlare-Ray “FRA” I get a TTFB of about 500ms-600ms

The response Header looks like this:

When I bypass CloudFlare (with curl) my TTFB is at 40ms.
Also when I call other sites which are static (like this one) my TTFB is very good and at about 30ms-40ms

Are there any issues with FRA or anything else what casues this?

On another Page I use Cache Everything and the TTFB also is very high.

Header:

Notics: this request is a very lightweight 304 and is a HIT in CloudFlare Cache. It varies very hard and even goes up to about 3s to respond.

[sdayman]

@M4rt1n

Also when I call other sites which are static (like this one) my TTFB is very good and at about 30ms-40ms

And those sites are also going through Cloudflare/FRA?

Are you and your server are in Germany?

Give the http://sitemeer.com/ test a try.

Update: I tried Sitemeer for one of my sites and TTFB is also in the 500ms range.

[M4rt1n]

@sdayman

And those sites are also going through Cloudflare/FRA?

Are you and your server are in Germany?

Give the http://sitemeer.com/ test a try.

1. No this other page is not getting routed through Cloudflare
2. Yes all my Servers are located in germany
3. Sitemeer: TTFB between 200ms-1000ms

But TTFB of 1000ms is normal for canada I guess.

#EDIT: pls see my first Post I added another test with “Cache Everything” and a HIT in Cache with about 760ms.
There have to be something at CF side as all Domains/Sites which are not getting roited through CF are working fast as expected

[sdayman]

Which Sitemeer location gave you ~200ms?

If you want to see how other sites in FRA perform, try this:
https://cloudflare-test.judge.sh/ (My LAX results are below)

As always, when it comes to routing, “Your mileage may vary”.

[M4rt1n]

@sdayman

Screen Shot 2020-05-02 at 9.07.51 AM2616×2502 386 KB

Even there I notice the second Ping is sometimes much higher then the first one, which is an indicator for heavy load.

@sdayman

Which Sitemeer location gave you ~200ms?

For something in italy. But this varies from call to call. I even get sometimes 140ms, but very often about 600ms. And once even 2.93s (see first post)

Here my results:

[sandro]

From your screenshot I’d say this is an issue between your ISP and the FRA PoP. You seem to be exclusively routed via FRA and there the latency figures of all sites are higher than they should be in this case.

[M4rt1n]

1. Higher latency is ok, normaly from AMS or LHR my TTFB (with Cache Everything) is at about 30ms-40ms. But 760ms or 2930ms (both with CF HIT) is just not acceptable.

2. If this problem is related to FRA, why does @sdayman’s screenshot (from LAX) also show Pings of about 600ms? And these are not the first Pings, but the second ones.

Between 40ms TTFB and 760 are 1900% difference… this means the performance of CF is actually dropping by 1900% and I would really like to know if this is temporary, if this is getting seen as an issue and will be fixed or how exactly this is getting handles internally?

Also: FRA (DE-CIX) is the most important and biggest Internet-Node in the world as far as I know.

So actually I just want to know WHY the performance actually is dropping by 1900%. The fact that it is dropping did I realised.

Also:

@sandro

I’d say this is an issue between your ISP and the FRA PoP

Thats exactly what I would not say as THIS exactly proofes the opposit. Aas its shows: no matter where you are meassuring, the TTFB (PING) is much much higher then expected. So this is not related to my ISP

At: CloudFlare Status everything in EU is getting the “operational” status, but “degraded performance” would really fit much more and even state that there is an issue or if this is getting inspected

[sandro]

@M4rt1n

Thats exactly what I did not say as THIS exactly proofes the opposit. Aas its shows: no matter where you are meassuring, the TTFB (PING) is much much higher then expected. So this is not related to my ISP

I am not sure how this proves the opposite. Every single site there has a relatively high latency figure. So that could very well point to an issue between your ISP and Cloudflare.

The interesting thing is, the second connections are always much faster.

One other thing, you mentioned you had on Sitemeer up to a second TTFB. The highest number I could find was 241 milliseconds. Are you sure you checked the right data?

[M4rt1n]

@sandro

I am not sure how this proves the opposite. Every single site there has a relatively high latency figure. So that could very well point to an issue between your ISP and Cloudflare.

Because all other sites are slow aswell. And they do not have anything to do with my ISP. Also if tested with the test tool this measures from different locations.

@sandro

One other thing, you mentioned you had on Sitemeer up to a second TTFB. The highest number I could find was 241 milliseconds. Are you sure you checked the right data?

Test some more times as this varys a lot.

@sandro

The interesting thing is, the second connections are always much faster.

Just look at this:

14 of 20 test have a (much) slower second TTFB compared to the first one.
How do you explain that?
That are a bit above 50%.
Also: this is from LAX, so nothing to do with my ISP

[sandro]

@M4rt1n

Because all other sites are slow aswell.

What are “all other sites”? If there is an issue between your ISP and FRA, there naturally will be a performance issue whenever you go via Cloudflare.

@M4rt1n

Test some more times as this varys a lot.

I looked at the numbers you tested yesterday

@M4rt1n

Just look at this:

That’s @sdayman’s screenshot, isnt it? Lets please focus on your connection issues.

[M4rt1n]

@sandro

That’s @sdayman’s screenshot, isnt it? Lets please focus on your connection issues.

But that is exactly what I say: this issue is not just here at my site, but also at his site. So how can this (same issue) be related to my ISP?
Also: all this “tests” doesnt matter that much, but when I call my site and do get a TTFB of about 700ms this matter a lot as my whole site is loading in just 200s-300ms.

Here another test which shows that this is related to FRA:

Even higher TTFB as if it does through Sydney.

Also when I test it with another page (same Server), also routed through FRA it gives me this:

TTFB Drop from 1.8s to 121ms. Not notice this:

TTFB Site 1: 1.8s (CF HIT)
TTFB Site 2: 121ms (CF DYNAMIC)

How is that evene possible when both pages are on the same Server? Use the same PoP (FRA)?
And how can this be related to my ISP if there is nothing done differently? Both .de Domains, Both same CF-Subscription. But on the same Server.

And if I not use “cache everything” I get a much slower site? That does not make sense, there is something wrong

[sandro]

@M4rt1n

this issue is not just here at my site, but also at his site. So how can this (same issue) be related to my ISP?

He is much further away. And as I said the number you mentioned cant be accurate as the highest TTFB was 241 milliseconds.

Look at the numbers of your site from a current check

Except for two, all of them are well below 100 milliseconds.

[M4rt1n]

Yes, sitemeer.com now shows good stats, but in real life they are not good at all.
Also the both test posted above are just made 2 mins before. So they are up to date, why do I get a TTFB of 1.8s there if sitemeer shows everything is very fast?

[M4rt1n]

@sandro

He is much further away

I dont know if he tested MY or HIS site?

[sandro]

@M4rt1n

but in real life they are not good at all.

These numbers are actual numbers

Based on your earlier screenshot I’d still say there is some connection issue between your ISP and FRA. The only puzzling thing is the second connection.

[sandro]

@M4rt1n

And if I not use “cache everything” I get a much slower site? That does not make sense, there is something wrong

You didnt mention that before. That could explain a lot, especially why the second request is faster.

[M4rt1n]

Sorry I mean: if I use Cache Everything the TTFB is slower.
So exactly the other ways around

The TTFB with 2.93s and 763ms are both with Cache Everything

[sdayman]

@M4rt1n

I dont know if he tested MY or HIS site?

The LAX results are just a Cloudflare performance test from my home. It tests a specific set of sites using Cloudflare. By testing the same sites, users can compare Cloudflare performance from different locations.

[M4rt1n]

Just wanted to let you guys know that since today the probem is gone. TTFB at 40ms-50ms. That is ok so far.

Thank you both @sdayman @sandro
Actually I dont know exactly what solved the problem, but I will keep this Thread updated if something changes.

[MarkMeyer]

I was doing some foo yesterday and it seems to me that their link at DE-CIX FRA either had some issues or a lot of traffic to handle, oder the router was busy. How ever. I had the same results yesterday via judge.sh. Up to 1.5 seconds. now it’s down to 293 (first) and 180 (second) there. I tested against a Nextcloud instance, where only a handful stuff is cached.

TCP Trace and MTRs from yesterday (slightly redacted):

mark@jumper:~$ tcptraceroute cloud.xxxxxxxxxx.de 443
Selected device ens160, address 172.30.0.43, port 35231 for outgoing packets
Tracing the path to cloud.tchnby.de (104.18.8.87) on TCP port 443 (https), 30 hops max
 1  172.30.0.1  0.493 ms  0.355 ms  0.287 ms
 2  xxx.xxx.24.3  0.801 ms  0.488 ms  0.415 ms
 3  gi7-2.xxxxxxxxxxx.net (xxx.xxx.200.77)  1.052 ms  0.932 ms  1.142 ms
 4  te0.xxxxxxxxxxx.net (xxx.xxx.249)  2.168 ms  1.591 ms  1.598 ms
 5  te0.xxxxxxxxxxx.net (xxx.xxx.249)  1.256 ms  1.129 ms  1.554 ms
 6  de-cix-frankfurt.as13335.net (80.81.194.180)  **62.090 ms  99.092 ms  77.305** ms
 7  104.18.8.87 [open]  **1.209 ms  1.422 ms**  0.989 ms


 Host                                                      Loss%   Snt   Last   Avg  Best  Wrst StDev
 1. _gateway                                                0.0%    19    0.3   0.3   0.2   0.4   0.1
 2. xxx.xxx.24.3                                            0.0%    19    0.4   0.6   0.4   1.1   0.2
 3. gi7xxxxxx.net                                           0.0%    18    1.2   1.0   0.7   1.6   0.2
 4. te0xxxxxx.net                                           0.0%    18    1.5   1.8   1.3   5.3   0.9
 5. te0xxxxxx.net                                           0.0%    18    1.2   1.4   1.1   1.9   0.2
 6. de-cix-frankfurt.as13335.net                           27.8%    18   55.9  83.0  46.8 152.8  32.7
 7. 104.18.9.87                                             0.0%    18    1.5   7.3   0.7  25.1   9.0


1. static-ip-85-xxx-xxx-xxx.inaddr.ip-pool.com               0.0%     9    1.5   0.2   0.1   1.5   0.4
 2. 92.204.12.10                                             0.0%     9    0.4   0.4   0.3   0.7   0.0
 3. ae3-u100.cr-vega.sxb1.bb.godaddy.com                     0.0%     8    0.4   2.8   0.4  12.1   4.2
 4. ae7.cr-antares.fra10.bb.godaddy.com                      0.0%     8    3.8   4.3   3.6   8.2   1.5
 5. ae2.cr-polaris.fra1.bb.godaddy.com                       0.0%     8    3.4   3.5   3.4   3.6   0.0
 6. de-cix-frankfurt.as13335.net                             12.5%     8   90.1  71.2   8.6  96.1  29.6
 7. 104.18.9.87                                              0.0%     8    3.6   3.7   3.6   3.8   0.0

See the lost packets? That indicates to me that there were some $issues. Whatever it was.

Today everything’s fine:

 Host                                Loss%   Snt   Last   Avg  Best  Wrst StDev
 1. _gateway                          0.0%   446    0.2   0.3   0.2   0.5   0.1
 2. xxx.xxx.24.3                      0.0%   445    0.5   1.9   0.3 167.7  13.3
 3. gi7xxxxxxx.net                    0.0%   445    1.2   1.0   0.8   1.8   0.2
 4. te0xxxxxxx.net                    0.0%   445    1.8   1.7   1.4   4.8   0.3
 5. te0xxxxxxx.net                    0.0%   445    1.2   1.3   1.0   4.1   0.3
 6. de-cix-frankfurt.as13335.net      0.0%   445    7.6   9.6   1.5 111.8  16.1
 7. 104.18.8.87                       0.0%   445    1.1   7.8   0.7  43.9  13.6