502 Bad Gateway between Cloudflare and our API server

Website, Application, Performance
1

Zone-ID: 453cc6cf15e139f96404695ced7372a3
Account-ID: 60fa0066b35ee998eb22129c1d154ac7
Domain: wonderlandengine.com (api.wonderlandengine.com)

We are experiencing an issue with one of our subdomains. On the subdomain a REST API endpoint is deployed that logs in users and handles requests. The API server is working without issues when testing it locally. Deployed to the server we receive “502 Bad Gateway” errors from Cloudflare.

The DNS configuration via Cloudflare is “DNS only”.

A api 185.26.156.222 DNS only
AAAA api 2a00:d0c0:200:0:f812:dff:fe10:bf34 DNS only

A test request is the following:

curl 'https://api.wonderlandengine.com/auth/signup' \
  -X 'OPTIONS' \
  -H 'authority: api.wonderlandengine.com' \
  -H 'accept: */*' \
  -H 'accept-language: en-GB,en-US;q=0.9,en;q=0.8' \
  -H 'access-control-request-headers: content-type' \
  -H 'user-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/114.0.0.0 Safari/537.36' \
  --compressed --verbose

With the following response:

Summary 
*   Trying 185.26.156.222:443...
* Connected to api.wonderlandengine.com (185.26.156.222) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
*  CAfile: /etc/ssl/certs/ca-certificates.crt
*  CApath: /etc/ssl/certs
* TLSv1.0 (OUT), TLS header, Certificate Status (22):
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.2 (IN), TLS header, Certificate Status (22):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (OUT), TLS header, Finished (20):
* TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.2 (OUT), TLS header, Certificate Status (22):
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.2 (IN), TLS header, Finished (20):
* TLSv1.2 (IN), TLS header, Certificate Status (22):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS header, Supplemental data (23):
* TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):
* TLSv1.2 (IN), TLS header, Supplemental data (23):
* TLSv1.3 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (IN), TLS header, Supplemental data (23):
* TLSv1.3 (IN), TLS handshake, CERT verify (15):
* TLSv1.2 (IN), TLS header, Supplemental data (23):
* TLSv1.3 (IN), TLS handshake, Finished (20):
* TLSv1.2 (OUT), TLS header, Supplemental data (23):
* TLSv1.3 (OUT), TLS handshake, Finished (20):
* SSL connection using TLSv1.3 / TLS_AES_256_GCM_SHA384
* ALPN, server accepted to use h2
* Server certificate:
*  subject: CN=api.wonderlandengine.com
*  start date: Jul  3 03:05:36 2023 GMT
*  expire date: Oct  1 03:05:35 2023 GMT
*  subjectAltName: host "api.wonderlandengine.com" matched cert's "api.wonderlandengine.com"
*  issuer: C=US; O=Let's Encrypt; CN=R3
*  SSL certificate verify ok.
* Using HTTP2, server supports multiplexing
* Connection state changed (HTTP/2 confirmed)
* Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0
* TLSv1.2 (OUT), TLS header, Supplemental data (23):
* TLSv1.2 (OUT), TLS header, Supplemental data (23):
* TLSv1.2 (OUT), TLS header, Supplemental data (23):
* Using Stream ID: 1 (easy handle 0x562e80c0de90)
* TLSv1.2 (OUT), TLS header, Supplemental data (23):
> OPTIONS /auth/signup HTTP/2
> Host: api.wonderlandengine.com
> accept-encoding: deflate, gzip, br, zstd
> authority: api.wonderlandengine.com
> accept: */*
> accept-language: en-GB,en-US;q=0.9,en;q=0.8
> access-control-request-headers: content-type
> user-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/114.0.0.0 Safari/537.36
> 
* TLSv1.2 (IN), TLS header, Supplemental data (23):
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* TLSv1.2 (IN), TLS header, Supplemental data (23):
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* old SSL session ID is stale, removing
* TLSv1.2 (IN), TLS header, Supplemental data (23):
* Connection state changed (MAX_CONCURRENT_STREAMS == 128)!
* TLSv1.2 (OUT), TLS header, Supplemental data (23):
* TLSv1.2 (IN), TLS header, Supplemental data (23):
* TLSv1.2 (IN), TLS header, Supplemental data (23):
< HTTP/2 502 
< server: nginx
< date: Mon, 10 Jul 2023 07:57:25 GMT
< content-type: text/html
< content-length: 552
< x-xss-protection: 1; mode=block
< x-frame-options: SAMEORIGIN
< strict-transport-security: max-age=31536000
< x-content-type-options: nosniff
< referrer-policy: strict-origin-when-cross-origin
< 
<html>
<head><title>502 Bad Gateway</title></head>
<body>
<center><h1>502 Bad Gateway</h1></center>
<hr><center>nginx</center>
</body>
</html>
<!-- a padding to disable MSIE and Chrome friendly error page -->
<!-- a padding to disable MSIE and Chrome friendly error page -->
<!-- a padding to disable MSIE and Chrome friendly error page -->
<!-- a padding to disable MSIE and Chrome friendly error page -->
<!-- a padding to disable MSIE and Chrome friendly error page -->
<!-- a padding to disable MSIE and Chrome friendly error page -->
* Connection #0 to host api.wonderlandengine.com left intact

Please advise, where we should look for the error. It appears to be an issue between Cloudflare and the server itself, the error “old SSL session ID is stale, removing” sounds like something certificate related is going on?

This is one of those “we didn’t change anything, but it stopped working” situations. Some days ago the 502 errors started to appear.

Is there anything in the Clouflare interface that we can use to debug the issue?

Edit: running dig on the domain shows three name servers, but configured are only two (the first two).

 dig  wonderlandengine.com

; <<>> DiG 9.18.12-0ubuntu0.22.04.2-Ubuntu <<>> wonderlandengine.com
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 33982
;; flags: qr rd ra; QUERY: 1, ANSWER: 3, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 65494
;; QUESTION SECTION:
;wonderlandengine.com.		IN	A

;; ANSWER SECTION:
wonderlandengine.com.	300	IN	A	104.26.11.190
wonderlandengine.com.	300	IN	A	104.26.10.190
wonderlandengine.com.	300	IN	A	172.67.69.23

;; Query time: 16 msec
;; SERVER: 127.0.0.53#53(127.0.0.53) (UDP)
;; WHEN: Mon Jul 10 15:19:08 +07 2023
;; MSG SIZE  rcvd: 97
3

hi @davidsneighbour, as per ​​502/504 from your origin web server

Cloudflare returns an Cloudflare-branded HTTP 502 or 504 error when your origin web server responds with a standard HTTP 502 bad gateway or 504 gateway timeout error.

Is there anything in the Clouflare interface that we can use to debug the issue?

You can check on the Cloudflare dashboard under Analytics & Logs > Traffic > Status Code (502).

I went ahead and looked up our internal dashboards for 502 on wonderlandengine.com, and there were no errors in the past 24 hours.

If you are pushing these logs and fields to your own storage(API server), are you able to share the log with all the fields you have captured, as we can use this to investigate further?

1 Like
4

Thank you for your answer. We fixed the issue that was , of course, on our side :slight_smile: Knowing now where to look for more information, we will get there faster next time.

One question, though: If we configured only two nameservers for the site, why comes dig up with three IP addresses? All of them are at Cloudflare so my guess is that this is some form of backup?