IP address of cloudflare server when displaying error message

Currently on the error page of cloudflare, we have the ray id, and the ip of the visitor that is displayed but not the ip address of the cloudflare server that issued the request to the server (that eventually failed)

It would be really critical to have this IP displayed on the error page, as it’s needed when we want to debug those kind of connection issues, and with just the cloudflare ray I tried to contact the support to get this information but got closed immediately with no human reply

What kind of an error have you got?

May I ask you to post your ticket number here?

And it never was displayed, as far I remember.

Is it really neccessarry to have it there?
If I am from country A, and have the CF server IP a, but you are from country B and have the CF server IP b and we both see the same error, who’s fault is it?

If the Cloudflare edge server is broken in some part of connection to the origin, it should switch to it’s neighbour or another one, while alerting Cloudflare team about that failure, shouldn’t it?

Depending on the error type, I would recommend to check out host origin/server log files, Cloudflare status page, hosting provider status page and Cloudflare Firewall events.

Furthermore, regarding Cloudflare errors, have a look on the below articles for more information:

1 Like

Here is my ticket ID 2204974

I have errors ranging from 500 to 525, which is often just a firewall issue (in this case fail2ban) but have already added all of the cloudflare ips listed publicly on the whitelist hence why I’m looking for the IP of the few failing requests to make sure that IP isn’t maybe some new datacenter that isn’t listed there

Regarding the logs on my server, that just supports my point, it’s impossible to query them without the cloudflare ip that failed because that would be the one that is logged. Without it, needle in a haystack

Either here or in the dashboard being able to see the failing ray ids to cross reference them without having to bother the support (but I feel like displaying them on this error page would just be a lot simpler to implement)

It’s a necessary information to have displayed at least somewhere, because to find the cause of the connection problem we need this IP address

Regarding 5xx errors:

Regarding the error 525, kindly check this two articles (one has the same title as above one, but is pointing to 525 error):

May I ask what SSL option (Flexible, Full, Strict) have you got selected under SSL/TLS tab on Cloudflare dashboard for your domain?

Therefore, have you got an valid SSL certificate installed on your host origin/server which covers both your main domain and other sub-domains?

Here is a way to re-check if correctly setup the SSL with Cloudflare:

To troubleshoot more about SSL issues:

Regarding available SSL options at Cloudflare dashboard, check here:

By this statement I assume you have successfully implemented the steps from below article?:

Cloudflare IP address list can be found on below article:

May I ask, but, have you saved your iptables on your server, or any other IDS to apply the changes?

If any requests passed to the origin, you should see the Cloudflare IP address there.

Do you have SSL enabled for your application at your Web server (Apache config or Nginx vhost file)?
Can you ping in/out?

Furthermore, is your app/web server working over 443 or some other port compatible HTTPS with Cloudflare one’s from the list below?:

What happens when you temporarly disable Cloudflare? (Pause or put to Development mode or make :orange: records to :grey:)

Just in case, there is a way to go with Cloudflare CA Origin Certificate using the steps from below to achieve Full (Strict) SSL :

I really hope we can resolve it here using suggestions and recommendation, in case not, maybe @cloonan could take a look at your ticket.

Sorry if my problem is not super clear, but pretty much there is no issue with 99.9% of the requests, I do have SSL enabled on the origin (with let’s encrypt provided by plesk) and cloudflare set to Full (strict)

I have indeed added all the ips listed there in fail2ban and yes it’s saved, with fail2ban it’s just a config file containing the list

[DEFAULT]
ignoreip = 127.0.0.1/8 2a01:4f8:13b:1d6::2 103.21.244.0/22 103.22.200.0/22 103.31.4.0/22 2400:cb00::/32 2606:4700::/32 2803:f800::/32 2405:b500::/32 2405:8100::/32 2a06:98c0::/29 2c0f:f248::/32 104.16.0.0/13 104.24.0.0/14 108.162.192.0/18 131.0.72.0/22 141.101.64.0/18 162.158.0.0/15 172.64.0.0/13 173.245.48.0/20 188.114.96.0/20 190.93.240.0/20 197.234.240.0/22 198.41.128.0/17

Before that the issue did happen a lot more and now it’s almost gone but a few still happen and it’s not an isolated issue as it does happen on a regular basis as shown from sentry and once the user hits the issue he will continue hitting it till the end of his session because he stays on the same cloudflare ip (this is a high traffic website with around 6000 unique visitors per day and 150 000 requests per day)
image
image

The error code varies from 500 to 525 but always is in fact the exact same problem where cloudflare is just denied connection to the server (So I’m unsure why it’s just not always error 520 but maybe there’s an explanation)

That’s not the issue, the issue is finding it in there, there is literally a million ip addresses in the logs, without knowing the IP that triggered the issue, I will never be able to find it. On top of that fail2ban only logs when the ip goes to the blacklist and not subsequent requests but finding it there is also not an option as there is no error message and more than 800 ips

Effectively the only thing I do have is the ray id and original ip of the visitor of all the failing requests but not the ip of cloudflare which is the only thing that would help me get closer to solve the issue

Here is an example of the error message the visitors are getting and being logged in sentry

<!DOCTYPE html>
<!--[if lt IE 7]> <html class="no-js ie6 oldie" lang="en-US"> <![endif]-->
<!--[if IE 7]>    <html class="no-js ie7 oldie" lang="en-US"> <![endif]-->
<!--[if IE 8]>    <html class="no-js ie8 oldie" lang="en-US"> <![endif]-->
<!--[if gt IE 8]><!--> <html class="no-js" lang="en-US"> <!--<![endif]-->
<head>
<meta http-equiv="refresh" content="0">

<title>xxxxxx.hu | 520: Web server is returning an unknown error</title>
<meta charset="UTF-8" />
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
<meta http-equiv="X-UA-Compatible" content="IE=Edge,chrome=1" />
<meta name="robots" content="noindex, nofollow" />
<meta name="viewport" content="width=device-width,initial-scale=1" />
<link rel="stylesheet" id="cf_styles-css" href="/cdn-cgi/styles/main.css" type="text/css" media="screen,projection" />


</head>
<body>
<div id="cf-wrapper">
    <div id="cf-error-details" class="p-0">
        <header class="mx-auto pt-10 lg:pt-6 lg:px-8 w-240 lg:w-full mb-8">
            <h1 class="inline-block sm:block sm:mb-2 font-light text-60 lg:text-4xl text-black-dark leading-tight mr-2">
              
              <span class="cf-error-type">Error</span>
              <span class="cf-error-code">520</span>
            </h1>
            <span class="inline-block sm:block font-mono text-15 lg:text-sm lg:leading-relaxed">Ray ID: 66ed674bd8c94dca &bull;</span>
            <span class="inline-block sm:block font-mono text-15 lg:text-sm lg:leading-relaxed">2021-07-14 20:12:01 UTC</span>
            <h2 class="text-gray-600 leading-1.3 text-3xl font-light">Web server is returning an unknown error</h2>
        </header>
        
        <div class="my-8 bg-gradient-gray">
            <div class="w-240 lg:w-full mx-auto">
                <div class="clearfix md:px-8">
                  
<div id="cf-browser-status" class=" relative w-1/3 md:w-full py-15 md:p-0 md:py-8 md:text-left md:border-solid md:border-0 md:border-b md:border-gray-400 overflow-hidden float-left md:float-none text-center">
  <div class="relative mb-10 md:m-0">
    <span class="cf-icon-browser block md:hidden h-20 bg-center bg-no-repeat"></span>
    <span class="cf-icon-ok w-12 h-12 absolute left-1/2 md:left-auto md:right-0 md:top-0 -ml-6 -bottom-4"></span>
  </div>
  <span class="md:block w-full truncate">You</span>
  <h3 class="md:inline-block mt-3 md:mt-0 text-2xl text-gray-600 font-light leading-1.3">Browser</h3>
  <span class="leading-1.3 text-2xl text-green-success">Working</span>
</div>

<div id="cf-cloudflare-status" class=" relative w-1/3 md:w-full py-15 md:p-0 md:py-8 md:text-left md:border-solid md:border-0 md:border-b md:border-gray-400 overflow-hidden float-left md:float-none text-center">
  <div class="relative mb-10 md:m-0">
    <span class="cf-icon-cloud block md:hidden h-20 bg-center bg-no-repeat"></span>
    <span class="cf-icon-ok w-12 h-12 absolute left-1/2 md:left-auto md:right-0 md:top-0 -ml-6 -bottom-4"></span>
  </div>
  <span class="md:block w-full truncate">Frankfurt</span>
  <h3 class="md:inline-block mt-3 md:mt-0 text-2xl text-gray-600 font-light leading-1.3">Cloudflare</h3>
  <span class="leading-1.3 text-2xl text-green-success">Working</span>
</div>

<div id="cf-host-status" class="cf-error-source relative w-1/3 md:w-full py-15 md:p-0 md:py-8 md:text-left md:border-solid md:border-0 md:border-b md:border-gray-400 overflow-hidden float-left md:float-none text-center">
  <div class="relative mb-10 md:m-0">
    <span class="cf-icon-server block md:hidden h-20 bg-center bg-no-repeat"></span>
    <span class="cf-icon-error w-12 h-12 absolute left-1/2 md:left-auto md:right-0 md:top-0 -ml-6 -bottom-4"></span>
  </div>
  <span class="md:block w-full truncate">xxxxxx.hu</span>
  <h3 class="md:inline-block mt-3 md:mt-0 text-2xl text-gray-600 font-light leading-1.3">Host</h3>
  <span class="leading-1.3 text-2xl text-red-error">Error</span>
</div>

                </div>
              
            </div>
        </div>

        <div class="w-240 lg:w-full mx-auto mb-8 lg:px-8">
            <div class="clearfix">
                <div class="w-1/2 md:w-full float-left pr-6 md:pb-10 md:pr-0 leading-relaxed">
                    <h2 class="text-3xl font-normal leading-1.3 mb-4">What happened?</h2>
                    <p>There is an unknown connection issue between Cloudflare and the origin web server. As a result, the web page can not be displayed.</p>
                </div>
              
                <div class="w-1/2 md:w-full float-left leading-relaxed">
                    <h2 class="text-3xl font-normal leading-1.3 mb-4">What can I do?</h2>
                          <h3 class="text-15 font-semibold mb-2">If you are a visitor of this website:</h3>
      <p class="mb-6">Please try again in a few minutes.</p>

      <h3 class="text-15 font-semibold mb-2">If you are the owner of this website:</h3>
      <p><span>There is an issue between Cloudflare's cache and your origin web server. Cloudflare monitors for these errors and automatically investigates the cause. To help support the investigation, you can pull the corresponding error log from your web server and submit it our support team.  Please include the Ray ID (which is at the bottom of this error page).</span> <a rel="noopener noreferrer" href="https://support.cloudflare.com/hc/en-us/articles/200171936-Error-520">Additional troubleshooting resources</a>.</p>
                </div>
            </div>
              
        </div>

        <div class="cf-error-footer cf-wrapper w-240 lg:w-full py-10 sm:py-4 sm:px-8 mx-auto text-center sm:text-left border-solid border-0 border-t border-gray-300">
  <p class="text-13">
    <span class="cf-footer-item sm:block sm:mb-1">Cloudflare Ray ID: <strong class="font-semibold">66ed674bd8c94dca</strong></span>
    <span class="cf-footer-separator sm:hidden">&bull;</span>
    <span class="cf-footer-item sm:block sm:mb-1"><span>Your IP</span>: 37.76.xx.xxx (masked by me)</span>
    <span class="cf-footer-separator sm:hidden">&bull;</span>
    <span class="cf-footer-item sm:block sm:mb-1"><span>Performance &amp; security by</span> <a rel="noopener noreferrer" href="https://www.cloudflare.com/5xx-error-landing" id="brand_link" target="_blank">Cloudflare</a></span>
   
  </p>
</div><!-- /.error-footer -->
    </div>
</div>
</body>
</html>

What I do now is they’re all requests from the Frankfurt data centre, so they should be covered by the rule 162.158.0.0/15, but they are clearly not, so I’m thinking one ip of one of the servers is not within this range and that’s what I’m trying to verify but can’t

1 Like

Hm, if I understand correctly what you did, but fail2ban is not the same as iptables.

Maybe some failure at a switch or have you tried reaching out to your server/hosting provider?
Maybe some firewall at datacenter blocking requests?

I know, I also have them whitelisted in the firewall (which is iptables on plesk)

I have not, I somehow doubt the issue is anywhere else than a blockage by some rule on the server, and without any trace of the issue it’s also extremely unlikely they would be able to help (it’s a dedicated server at hetzner, which has proven extremely reliable in 6 years of service with not more than 4 minutes of downtime 2 years ago when one switch failed)

1 Like

I also do have servers at this hosting provider, true for the stated as far as for now :wink:

Regarding the hosting provider, may I ask are you using Hetzner Firewall templates from their Robot interface, or not, in this particular case for your dedicated server?

And you mentioned Plesk panel, well I am not as much familiar with using it so I am afraid I cannot help a lot with it.

Due to applied changes, have you tried restarting some services like PHP, web server (Nginx or Apache), or even networking service ?

Kindly and patiently wait for someone else to reply with more help.


The firewall on hetzner side is disabled

And yes the server was restarted somewhere around 2 weeks ago which is after the changes were applied

Thanks for your help :wink:

1 Like

This topic was automatically closed 15 days after the last reply. New replies are no longer allowed.