Geolocation Accuracy


#1

I have been using the Geolocation provided for Cloudflare for awhile now but recently noticed issues when I started to compare results to GeoLite2 from Maxmind, which is free.

MM = GeoLite2 Maxmind
CF = Cloudflare Geolocation

Examples
2001:470:733f:0:b843:95fd:2404:137d
MM - US < correct
CF - DE

It seems like MaxMind even on a free level is beating Cloudflare in Geo-location detection. Even when the underlying IP is the same MM is correct and CloudFlare is wrong. What I do not understand is why would Cloudflare be wrong if the recorded server IP and HTTP_CF_CONNECTING_IP are the same. Same with IP4/6 no difference.

Anyone have any experience with CloudFlare Geolocation Accuracy?

Anyone using alternative solutions?

Thanks!
Grant


#2

I’d also like to add that HTTP_CF_CONNECTING_IP is also returning a lot of

(DCH) Data Center/Web Hosting/Transit

type IP addresses rather then end user IP addresses

example:
107.167.102.183
66.102.7.26
141.0.12.20
64.233.172.216

Thankfully I am not relying on the above and also getting my own IP results. What often happens is a result like IP,107.167.108.173

Note there is a comma “,” separating two ip addresses.

The 1st is the Client IP and the second is the DCH transit IP.

The reason I mention all of this is because if the IP address returned by Cloudflare is a DCH IP then it is not reflecting the user. As a result cloudflare geolocation also sends over

city: Mountain View
country: US

when really
city: Accra
country: Ghana

very big difference!

It seems if anyone decides to use a VPN or Proxy, cloudflare only passes through the VPN/Proxy IP address.


#3

Hurricane Electric is based in the US, but that IP appears to be assigned to a tunnel in Frankfurt, Germany. The ultimate end user may or may not be in Germany but they’re probably close.


#4

Expanding Further:
Note this is by no means a one off scenario. I have 5k+ results like this in my database.

IP Address Details: 178.241.75.218
Country: TR TR
City: Eskişehir
Region: Eskişehir
Hostname: 178.241.75.218
ISP Turkcell
Organization Turkcell
ASN 16135
Proxy/VPN Detection: false
VPN: false
TOR: false
Fraud Score 0
Recent Abuse: false
Time Zone Europe/Istanbul

IP Address Details: 107-178-44-153
Country: US US
City: Cupertino
Region: California
Hostname: 107-178-44-153.IP.cloudmosa.com
ISP CloudMosa
Organization CloudMosa
ASN 6939
Proxy/VPN Detection: true
VPN: true
TOR: false
Fraud Score 75
Recent Abuse: false
Time Zone America/Los_Angeles

Again I still do not understand why Cloudflare reports back US when the user is actually in Turkey. The HTTP_CF_IPCOUNTRY data provided by Cloudflare is giving me horribly incorrect results.


#5

I’d love to know how or if anyone is using this in a production capacity? Or if they are just using this as a guide?

Thanks,
Grant


#6

Production Warning
If you use either CF_CONNECTING_IP or CF_IPCOUNTRY do not expect production level accuracy without making some modifications to general install instructions. Specifically if you do not want your routing to happen based on proxy IP results from Cloudflare, routing based on fake geolocations from Cloudflare and lastly storing incorrect results in a database for future processing.

Step One - CloudFlare Setup
#save in /etc/nginx/cloudflare.conf
set_real_ip_from 103.21.244.0/22;
set_real_ip_from 103.22.200.0/22;
set_real_ip_from 103.31.4.0/22;
set_real_ip_from 104.16.0.0/12;
set_real_ip_from 108.162.192.0/18;
set_real_ip_from 131.0.72.0/22;
set_real_ip_from 141.101.64.0/18;
set_real_ip_from 162.158.0.0/15;
set_real_ip_from 172.64.0.0/13;
set_real_ip_from 173.245.48.0/20;
set_real_ip_from 188.114.96.0/20;
set_real_ip_from 190.93.240.0/20;
set_real_ip_from 197.234.240.0/22;
set_real_ip_from 198.41.128.0/17;
set_real_ip_from 2400:cb00::/32;
set_real_ip_from 2405:8100::/32;
set_real_ip_from 2405:b500::/32;
set_real_ip_from 2606:4700::/32;
set_real_ip_from 2803:f800::/32;
set_real_ip_from 2c0f:f248::/32;
set_real_ip_from 2a06:98c0::/29;
real_ip_header X-Forwarded-For;
real_ip_recursive on;

Note that we are using “X-Forwarded-For” here versus the suggested “CF-Connecting-IP” as outlined in the Cloudflare setup guide. We also add real_ip_recursive on to help with detecting the clients real ip address.

Step Two - Website’s Nginx Setup
if ($http_x_forwarded_for ~ “^([^,]+)” ) {
set $first_xff $1;
}

This captures the first element from X-Forwarded-For and gives us the clients real IP address. This is also easy to get in PHP without doing the above but if you want to utilize the real IP for geolocation checks in nginx this is critical.

Step Three - Setup Maxmind GeoLite2 Free
Firstly this is free, secondly this is extremely accurate on a country level. As an added bonus it also provides city, state, latitude, longitude, and postal with some accuracy. I typically focus on country/city so for my usage the results are pretty accurate.

Get the database files
http://geolite.maxmind.com/download/geoip/database/GeoLite2-City.tar.gz
http://geolite.maxmind.com/download/geoip/database/GeoLite2-Country.tar.gz

Next part requires a updates to nginx, specifically to install the geoip2 module. Note that I initially tried doing this by just adding the module but ended with conflicts. As a result I found that I recomplinging nginx again worked.

./configure --prefix=/etc/nginx --with-cc-opt=’-g -O2 -fdebug-prefix-map=/build/nginx-K5HqCe/nginx-1.12.1=. -fstack-protector-strong -Wformat -Werror=format-security -fPIC -Wdate-time -D_FORTIFY_SOURCE=2’ --with-ld-opt=’-Wl,-Bsymbolic-functions -Wl,-z,relro -Wl,-z,now -fPIC’ --prefix=/usr/share/nginx --conf-path=/etc/nginx/nginx.conf --http-log-path=/var/log/nginx/access.log --error-log-path=/var/log/nginx/error.log --lock-path=/var/lock/nginx.lock --pid-path=/run/nginx.pid --modules-path=/usr/lib/nginx/modules --http-client-body-temp-path=/var/lib/nginx/body --http-fastcgi-temp-path=/var/lib/nginx/fastcgi --http-proxy-temp-path=/var/lib/nginx/proxy --http-scgi-temp-path=/var/lib/nginx/scgi --http-uwsgi-temp-path=/var/lib/nginx/uwsgi --with-debug --with-pcre-jit --with-http_ssl_module --with-http_stub_status_module --with-http_realip_module --with-http_auth_request_module --with-http_v2_module --with-http_dav_module --with-http_slice_module --with-threads --with-http_addition_module --with-http_geoip_module=dynamic --with-http_gunzip_module --with-http_gzip_static_module --with-http_sub_module --with-stream=dynamic --with-stream_ssl_module --with-mail=dynamic --with-mail_ssl_module --with-http_secure_link_module --with-http_mp4_module --add-dynamic-module=/usr/src/ngx_http_geoip2_module

Step Four - Update Nginx.conf
#Load above http
load_module modules/ngx_http_geoip2_module.so;

#Load Country and City detection GEOLITE2
These go inside http

    include /etc/nginx/cloudflare.conf; #FROM EARLIER
    geoip2 /mnt/RAM_disk/GeoLite2-Country.mmdb {
       $geoip2_data_country_code source=$first_xff country iso_code;
       $geoip2_data_continent_code source=$first_xff continent code;
    }

    geoip2 /mnt/RAM_disk/GeoLite2-City.mmdb {
       $geoip2_data_city_name source=$first_xff default=London city names en;
       $geoip2_data_state source=$first_xff subdivisions 0 iso_code;
       $geoip2_data_accuracy source=$first_xff location accuracy_radius;
       $geoip2_data_lat source=$first_xff default=0 location latitude;
       $geoip2_data_lng source=$first_xff default=0 location longitude;
       $geoip2_data_postal source=$first_xff default=0 postal code;
    }

Notice I use ram disk, while I am not sure if this helps entirely as Linux caches files.

Step Five - Update website conf
#add the if statement mentioned earlier to get the clients true IP address

if ($http_x_forwarded_for ~ "^([^,]+)" ) {
    set $first_xff $1;
}

Lastly add the following where required:

    fastcgi_param GEOIP_CITY_CONTINENT_CODE $geoip2_data_continent_code;
    fastcgi_param GEOIP_COUNTRY_CODE $geoip2_data_country_code;
    fastcgi_param GEOIP_STATE $geoip2_data_state;
    fastcgi_param GEOIP_CITY $geoip2_data_city_name;
    fastcgi_param GEOIP_ACCURACY $geoip2_data_accuracy;
    fastcgi_param GEOIP_LATITUDE $geoip2_data_lat;
    fastcgi_param GEOIP_LONGITUDE $geoip2_data_lng;
    fastcgi_param GEOIP_POSTAL $geoip2_data_postal;

Restart Nginx, and run a few tests. You can access the above simply by using:
_SERVER['GEOIP_CITY'].', '._SERVER[‘COUNTRY_CODE’];

Final Results
I have moved away from using Cloudflare’s geolocation detection and moved to a server based detection method which provides the country and city information in under 1 ms based on benchmarks. While also being accurate within the limitations of the database’s accuracy. This could be improved further by purchasing GeoIP2 databases.

Note also that you can also complete an IP lookup using php and the same database file expect lookup times to take around 300 ms give or take your setup.


#7

This is a very nice article. Do you have plan to write about using IP2Location LITE database?

Our company has a subscription of the IP2Location commercial database which has the data center range (DCH) under usage type. If you share how to use the free IP2Location LITE, we can use it immediately.

Thank you.


#8

Seems IP2Location has a similar module.


#9

This topic was automatically closed after 14 days. New replies are no longer allowed.