Since 1am on Aug 15 2023, something odd happened, one of our AWS customers started to have 504 errors with our PROXIED load balancer.
The traffic is coming from their AWS VPC to our load balancer that is pointed to our on-premises firewall. Just a plain virtual server with the normal nat on the firewall, and waf rules on cloudflare.
AWS → Cloudflare Proxied Load Balancer → Firewall WAN ports.
The 504 error is NOT present on our dashboard, so seems to be something between AWS and Cloudflare.
We created a different DNS record NOT proxied and NOT load balanced, and the client now works normally.
The service was working normally prior to this, we cannot pin point what happened. In our end, we do get error 499 on the dashboard, but not the 504.