Latency increases on proxying

I have an nginx running ec2 hosted in ap-south region

http get with dns_only
india -> ec2 takes 5ms
virginia -> ec2 takes 368ms

https get with dns_only (ssl certificate is served from ec2 nginx)
india -> ec2 takes 19ms
virginia -> ec2 takes 748ms

https get with dns proxied (ssl certificate is served from edge cloudfare server) (ssl full config is used)
india -> ec2 takes 552ms
virginia -> ec2 takes 851ms

domain A name: test.asyncify.com

question 1: why is it that after proxying the request takes more time for india to india communication ?
question 2: isnt using edge certificate supposed to bring down latency, or have i mis configured something ?

even im not able to reuse ssl sessions, each attempt establishes a new one

@sdayman can you please look into this and help, thanks

Hi @aditya999123, are you still encountering issues with this?

yes @cloonan, im still getting same results

I see support is finding 521 errors on their tests, do you see the errors in the origin logs?

nope, i have an nginx behind this

curl https://test.asyncify.com/xyz
should return {“message”:“xyz”}

tried with a different new nginx setup as well

test3.asyncify.com dns_only
test4.asyncify.com is proxied
both pointing to same ip 3.6.205.155
ssl config is full

nginx conf

server {
        listen 80;
        listen 443 default_server ssl;

        # ssl on;
    ssl_certificate /etc/letsencrypt/live/test3.asyncify.com/fullchain.pem; # managed by Certbot
    ssl_certificate_key /etc/letsencrypt/live/test3.asyncify.com/privkey.pem; # managed by Certbot

        server_name test3.asyncify.com test4.asyncify.com;

        location /ready {
                return 200 '{"message": "nginx is ready"}';
                add_header Content-Type text/plain;
        }
}
1 Like

On test2 I see a certificate error that I don’t see on test, but don’t understand the latency.

Wondering if any @MVP can throw an eye on this for deeper digging?

Unless you’re on Cloudflare Business plan or higher, you won’t hit Cloudflare’s Indian datacenters on lower paid or free tier plans and have Indian visitor requests directed most likely to Cloudflare’s Singapore datacenters due to Indian ISP/data costs and peering agreements in place.

For each of your test domain configs try curl check from an Indian location for domain CDN trace url and see what that brings back for colo (datacenter location the request was served from) when behind CF proxy

curl -s https://test.asyncify.com/cdn-cgi/trace

also try a trace from Indian location

traceroute test.asyncify.com

though keycdn performance test can also show same things https://tools.keycdn.com/performance?url=https://test.asyncify.com/

Indian requests are being routed to Cloudflare’s Singapore datacenter with cf-ray code = SIN

So goes back to for Indian location, you’d need Cloudflare Business or higher paid plans to get Indian visitors directed to Cloudflare Indian datacenters but that still isn’t a guarantee - just more likely to hit Indian datacenters. The higher the paid plan, the more likely you’d hit them i.e. you need Cloudflare Enterprise plan

But how are you measuring those numbers ? How and where you’re measuring from matters too as the TLS handshake is 2-way

1 Like

yes @cloonan there was some issue with test2
so i went ahead did a new setup on these

test3.asyncify.com dns_only
test4.asyncify.com is proxied
both pointing to same ip 3.6.205.155
ssl config is full

1 Like

@eva2000 im limiting my test to these two subdomains now, from an indian location only

test3.asyncify.com dns_only
test4.asyncify.com is proxied
both pointing to same ip 3.6.205.155
ssl config is full

$ curl -s https://test3.asyncify.com/cdn-cgi/trace
<html>
<head><title>404 Not Found</title></head>
<body bgcolor="white">
<center><h1>404 Not Found</h1></center>
<hr><center>nginx/1.14.0 (Ubuntu)</center>
</body>
</html>

$ curl -s https://test4.asyncify.com/cdn-cgi/trace
fl=35f290
h=test4.asyncify.com
ip=13.235.70.131
ts=1595566157.875
visit_scheme=https
uag=curl/7.58.0
colo=SIN
http=http/2
loc=IN
tls=TLSv1.2
sni=plaintext
warp=off 

yes it is being routed from singapore, however 550ms still seems very high, as the rtt from mumbai to singapore is about 57ms. nut yes this does explain a lot, thanks

  1. i dont follow about the likeliness to hit indian server, how does the likeliness work with plan or what factors does it depend on ?, because adding 500ms delay to existing applications is definitely not an option for us . please share more information on this

  2. is there a way to optimize 2nd tls handshake ? without argo

  • a) does edge and origin persist ssl session for multiple clients ?
    meaning
    clienta -> edge.india -> session(1) -> orign
    clientb -> edge.india -> session(x) -> orgin
    can x be 1 ?
  1. and if argo is the only option to optimize 2nd handshake, is there a way for argo to not become spof ?

How are you measuring 550ms ? as TLS handshake is 2 way, you can only optimise your end i.e. origin nginx web server for optimal TLS protocol i.e. TLSv1.3 and SSL ciphers and use ECDSA instead of traditional RSA 2048bit SSL certificates on origin nginx HTTPS i.e. for my Centmin Mod LEMP stack’s Nginx web server users to optimise connection between CF edge server and Centmin Mod Nginx origin servers https://community.centminmod.com/threads/improving-cloudflare-connections-to-origin-server-use-ecdsa-ssl-certs.14817/

See if that improves the TLS handshake side of things

Argo optimizes the server connection response time not the TLS handshake directly. But still improves things but at 550+ ms it’s more about geography + origin web server I suspect.

im measuring using curl commands from a different ec2 hosted in mumbai region

BUILD-MACHINE:~ $ cat curl_format2.txt 
HTTP Version:                     HTTP/%{http_version}\n
Header Size:                      %{size_header}\n
Request Size:                     %{size_request}\n
Download Size:                    %{size_download}\n
DNS Lookup (time_namelookup):     %{time_namelookup}\n
TCP Connect (time_connect):       %{time_connect}\n
SSL Handshake (time_appconnect):  %{time_appconnect}\n
Pre-Transfer (time_pretransfer):  %{time_pretransfer}\n
Redirect Time (time_redirect):    %{time_redirect}\n
TTFB (time_starttransfer):        %{time_starttransfer}\n
Time Total:                       %{time_total}\n
BUILD-MACHINE:~ $ curl -w "@curl_format2.txt" -v --trace-time "https://test4.asyncify.com/ready"

hurray, after enabling tls1.3 from nginx the latency dropped to 380ms,

however changing certificate had no effect on latency, but occasionally 1 in 10 requests would take 280ms

also it’d help a lot if you can share details on factors on which it is decided where the request would be proxied from, asking because our production account is on a business plan and would like to know if changing to enterprise would actually benefit us

looks like a good improvement

Enterprise is more likely to hit Indian datacenters but can vary. You can see sites on each CF plan and which datacenters they hit at https://cloudflare-test.judge.sh/. If you are in Indian location visiting the page at https://cloudflare-test.judge.sh/ will report for that moment in time which CF datacenter you get routed to

Also your origin server hardware/cpu model can make a difference too. I did a 13-way VPS benchmark comparison comparing servers from Upcloud vs DigitalOcean vs Linode vs Vultr vs Hetzner at https://community.centminmod.com/threads/13-way-vps-server-benchmark-comparison-tests-upcloud-vs-digitalocean-vs-linode-vs-vultr-vs-hetzner.17742/ and you can see performance can vary depending on server hardware/web host

Another thread I posted highlighting differences between cpus even with same web host https://community.centminmod.com/threads/amd-epyc-cpus-come-to-upcloud-vps-provider.20060/

So test your origin performance itself

This topic was automatically closed after 30 days. New replies are no longer allowed.