Intermittent ETIMEDOUT when using Cloudflare proxying

I have two Node.js applications that have CNAME records on Cloudflare with proxying turned on — let’s call them IngressApp and InternalApp.

My API users make HTTP requests to IngressApp and then IngressApp sends the request to InternalApp to do stuff.

Intermittently, when IngressApp tries to send a request to InternalApp I see the following error messages in the IngressApp logs:

Error: connect ETIMEDOUT 172.67.169.58:443
    at TCPConnectWrap.afterConnect [as oncomplete] (node:net:1494:16)
    at TCPConnectWrap.callbackTrampoline (node:internal/async_hooks:130:17)
500 Ran GET endpoint at /v0/ingress/feature in 127192ms. Input: {"0":"feature"} | Response: {"message":"connect ETIMEDOUT 172.67.169.58:443","code":"ETIMEDOUT","error":"Unhelpful error message here about an unidentified error","requestId":"xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx","statusCode":500}

When this error occurs there is no corresponding log in the InternalApp logs, and nothing shows up in the Cloudflare Dashboard’s Firewall section under Firewall Events. Also, strangely, all of these timeouts happen after exactly 127 seconds (give or take a few ms).

Turning off Cloudflare proxying seems to fix this issue, but I would strongly prefer to leave proxying on because of the tremendous benefits that Cloudflare proxying provides.

You’ll find the full Axios error below in case it’s useful, please help!

{
   errno: -110,
   code: 'ETIMEDOUT',
   syscall: 'connect',
   address: '104.21.79.74',
   port: 443,
   config: {
     transitional: {
       silentJSONParsing: true,
       forcedJSONParsing: true,
       clarifyTimeoutError: false
     },
     adapter: [Function: httpAdapter],
     transformRequest: [ [Function: transformRequest] ],
     transformResponse: [ [Function: transformResponse] ],
     timeout: 0,
     xsrfCookieName: 'XSRF-TOKEN',
     xsrfHeaderName: 'X-XSRF-TOKEN',
     maxContentLength: -1,
     maxBodyLength: -1,
     validateStatus: [Function: validateStatus],
     headers: {
       Accept: 'application/json, text/plain, */*',
       'Content-Type': 'application/json',
       'super-secret-key': 'MrAstleyWasHere',
       'User-Agent': 'axios/0.24.0',
       'Content-Length': 395
     },
     baseURL: 'https://internal.myapp.com',
     method: 'post',
     url: '/feature',
     data: '{"firstName":"Sarah","lastName":"Smith"}'
   },
   request: <ref *1> Writable {
     _writableState: WritableState {
       objectMode: false,
       highWaterMark: 16384,
       finalCalled: false,
       needDrain: false,
       ending: false,
       ended: false,
       finished: false,
       destroyed: false,
       decodeStrings: true,
       defaultEncoding: 'utf8',
       length: 0,
       writing: false,
       corked: 0,
       sync: true,
       bufferProcessing: false,
       onwrite: [Function: bound onwrite],
       writecb: null,
       writelen: 0,
       afterWriteTickInfo: null,
       buffered: [],
       bufferedIndex: 0,
       allBuffers: true,
       allNoop: true,
       pendingcb: 0,
       constructed: true,
       prefinished: false,
       errorEmitted: false,
       emitClose: true,
       autoDestroy: true,
       errored: null,
       closed: false,
       closeEmitted: false,
       [Symbol(kOnFinished)]: []
     },
     _events: [Object: null prototype] {
       response: [Function: handleResponse],
       error: [Function: handleRequestError]
     },
     _eventsCount: 2,
     _maxListeners: undefined,
     _options: {
       maxRedirects: 21,
       maxBodyLength: 10485760,
       protocol: 'https:',
       path: '/feature',
       method: 'POST',
       headers: [Object],
       agent: undefined,
       agents: [Object],
       auth: undefined,
       hostname: 'internal.myapp.com',
       port: null,
       nativeProtocols: [Object],
       pathname: '/feature'
     },
     _ended: false,
     _ending: true,
     _redirectCount: 0,
     _redirects: [],
     _requestBodyLength: 395,
     _requestBodyBuffers: [ [Object] ],
     _onNativeResponse: [Function (anonymous)],
     _currentRequest: ClientRequest {
       _events: [Object: null prototype],
       _eventsCount: 7,
       _maxListeners: undefined,
       outputData: [],
       outputSize: 0,
       writable: true,
       destroyed: false,
       _last: true,
       chunkedEncoding: false,
       shouldKeepAlive: false,
       maxRequestsOnConnectionReached: false,
       _defaultKeepAlive: true,
       useChunkedEncodingByDefault: true,
       sendDate: false,
       _removedConnection: false,
       _removedContLen: false,
       _removedTE: false,
       strictContentLength: false,
       _contentLength: 395,
       _hasBody: true,
       _trailer: '',
       finished: false,
       _headerSent: true,
       _closed: false,
       socket: [TLSSocket],
       _header: 'POST /feature HTTP/1.1\r\n' +
         'Accept: application/json, text/plain, */*\r\n' +
         'Content-Type: application/json\r\n' +
         'super-secret-key: MrAstleyWasHere\r\n' +
         'User-Agent: axios/0.24.0\r\n' +
         'Content-Length: 395\r\n' +
         'Host: internal.myapp.com\r\n' +
         'Connection: close\r\n' +
         '\r\n',
       _keepAliveTimeout: 0,
       _onPendingData: [Function: nop],
       agent: [Agent],
       socketPath: undefined,
       method: 'POST',
       maxHeaderSize: undefined,
       insecureHTTPParser: undefined,
       joinDuplicateHeaders: undefined,
       path: '/feature',
       _ended: false,
       res: null,
       aborted: false,
       timeoutCb: null,
       upgradeOrConnect: false,
       parser: null,
       maxHeadersCount: null,
       reusedSocket: false,
       host: 'internal.myapp.com',
       protocol: 'https:',
       _redirectable: [Circular *1],
       [Symbol(kCapture)]: false,
       [Symbol(kBytesWritten)]: 0,
       [Symbol(kNeedDrain)]: false,
       [Symbol(corked)]: 0,
       [Symbol(kOutHeaders)]: [Object: null prototype],
       [Symbol(errored)]: null,
       [Symbol(kUniqueHeaders)]: null
     },
     _currentUrl: 'https://internal.myapp.com/feature',
     [Symbol(kCapture)]: false
   },
   response: undefined,
   isAxiosError: true,
   toJSON: [Function: toJSON]
 }
2 Likes

I have the same problem, I didn’t know it was the proxy

UPDATE: Turning the proxy off does not fix the issue. Turning the proxy off results in a Origin DNS error error from Cloudflare.

Agreed, facing the same issue for a while. Removing Cloudflare completely from the path resolves the ETIMEOUT issue and the observation is that proxy only gives a different form of an error of Origin DNS.

We are operating in the India region.

We’re also encountering this issue with intermittent ETIMEDOUT errors when using Cloudflare proxy services. Interestingly, the problem disappears when Cloudflare is completely removed from the setup, Also “proxy only” is causing a different error related to Origin DNS. We are also centered in the Indian region.