Random high latency since 2014.10.24T20:20 CET

What is the name of the domain?

What is the issue you’re encountering

Our performance monitors are flickering in red because a large but random portion of requests suffer from very slow (250ms connect, 250ms SSL then +500ms to first byte than usual) connections

What steps have you taken to resolve the issue?

Tested from different locations (Datadog in both AWS and GCP regions), and Pingdom from another region to double-check.
Healthcheck Insights within Cloudflare show a steady <50ms to Origin.

Cloudflare status page is silent but everything suggests an issue with CDN. The slow SSL handshake seems to point finger at it.

What are the steps to reproduce the issue?

Test browser every minute.
Response time : 80% under 600ms. 20% between 2000 and 3000ms.

1 Like

Exact same issue on our side, we observe a latency increase on all our services since ~2014.10.24T20:00 CET

The sudden increase can be seen here :

Since then p50 is somehow stable but p90 is through the roof. Seems to affect randomly.

We have been experiencing a similar drastic increase in latency since yesterday (2024-10-24) - this issue is affecting only domains and endpoints that are proxied through Cloudflare.

A graph for our side because an image is worth a billion words

Using apache benchmark

ab -n [number of samples] [url]

On my website:

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:       23   38 106.3     28    1090
Processing:   119  140  12.5    138     207
Waiting:       85   99  13.3     95     173
Total:        144  179 108.0    166    1240

Percentage of the requests served within a certain time (ms)
  50%    166
  66%    171
  75%    172
  80%    175
  90%    185
  95%    198
  98%    234
  99%   1240
 100%   1240 (longest request)

Same laptop, on microsoft.com:

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:       20   24   4.0     23      48
Processing:    68  101  24.3     94     179
Waiting:       33   55  15.7     49     106
Total:         87  125  25.1    118     203

Percentage of the requests served within a certain time (ms)
  50%    118
  66%    128
  75%    137
  80%    142
  90%    163
  95%    185
  98%    195
  99%    203
 100%    203 (longest request)

Seems like latencies are back to normal… at exactly 20:20 CET today. That’s precisely 24h hours lasting incident.

It looks like it’s still happening on our side. According to our supervision, it started on the 25th at 8:20AM UTC+2 (Paris Time).

Is anyone else experiencing the same behavior? We haven’t made any changes to our infrastructure or cloudflare options.

Requests take a long time to reach our reverse proxies. By going direct (without cloudflare proxy), the problem disappears.

This topic was automatically closed after 15 days. New replies are no longer allowed.