~0.5% of JSON POST requests do not reach our server (through Cloudflare proxy)

Approximately 0.1% of JSON POST requests made from client browser to our website are not receiving a response from our website or, we assume, from Cloudflare. This has been happening for quite some time, so I recently upgraded to the PRO plan but that hasn’t solved it. I’ve spent countless hours reading and searching and am posting here for help.

Setup

  • Client -> Cloudflare (proxy) -> Rails App on Heroku. Our application is largely an angularjs app, and the failing requests (that we are aware of) are JSON POST requests.
  • Followed the Configure Cloudflare and Heroku over HTTPS guide. Triple checked everything.
  • Reviewed all configurations on Cloudflare to ensure compatibility between Cloudflare / Heroku. For example, Heroku doesn’t currently support TLS 1.0 or 1.1 so I’ve set the minimum version on Cloudflare to 1.2. No changes to config settings have made any impact on this issue.

Issue and Steps Taken to Resolve

In short, it seems that the request cannot reach Cloudflare. If the issue was between Cloudflare and Heroku, I would expect a 5** response or for Cloudflare to respond with some other timeout error message (correct?). However, the POST request receives an empty response with status of “-1”.

Confirming with BugSnag

After user reports of data not saving, we installed BugSnag on the front end and wrote code to retry requests. The workflow looks like this:

  1. Send JSON POST requests to our_endpoint.json.
  2. If request fails, retry two more times. Wait 7 seconds between requests. If any subsequent request succeeds, send an error report to BugSnag with all data.
  3. If all three attempts fail, send an error report (with the requests and responses) to BugSnag.

From our logs, the responses (in javascript) are in the format: { "data": null, "status": -1, ... }. The contain none of the Cloudflare headers or any other helpful information. To me this indicates that the request is not reaching Cloudflare’s endpoints.

Any help would be GREATLY appreciated!