Since a while I am facing intermittent TLS errors when accessing websites behind Cloudflare via IPv6. Of course I recognised it first for our website, but after searching for similar issues here, I found the same to happen on other websites behind Cloudflare, like https://symfony.com, https://repo.homebridge.io and others. Roughly in 50% of attempts, the error is thrown, else the connection works well. Also, there are days/hours where I cannot replicate the issue, and other days, where it happens more often. When I bypass Cloudflare it works all well. Here some output, using different console tools:
root@micha:~# curl -vI https://symfony.com
* Host symfony.com:443 was resolved.
* IPv6: 2606:4700:20::681a:4fb, 2606:4700:20::681a:5fb, 2606:4700:20::ac43:4826
* IPv4: 172.67.72.38, 104.26.4.251, 104.26.5.251
* Trying [2606:4700:20::681a:4fb]:443...
* Connected to symfony.com (2606:4700:20::681a:4fb) port 443
* ALPN: curl offers h2,http/1.1
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* CAfile: /etc/ssl/certs/ca-certificates.crt
* CApath: /etc/ssl/certs
* Recv failure: Connection reset by peer
* OpenSSL SSL_connect: Connection reset by peer in connection to symfony.com:443
* Closing connection
curl: (35) Recv failure: Connection reset by peer
root@micha:~# curl -vI --tls-max 1.2 https://symfony.com
* Host symfony.com:443 was resolved.
* IPv6: 2606:4700:20::681a:5fb, 2606:4700:20::ac43:4826, 2606:4700:20::681a:4fb
* IPv4: 104.26.4.251, 104.26.5.251, 172.67.72.38
* Trying [2606:4700:20::681a:5fb]:443...
* Connected to symfony.com (2606:4700:20::681a:5fb) port 443
* ALPN: curl offers h2,http/1.1
* TLSv1.2 (OUT), TLS handshake, Client hello (1):
* CAfile: /etc/ssl/certs/ca-certificates.crt
* CApath: /etc/ssl/certs
* Recv failure: Connection reset by peer
* OpenSSL SSL_connect: Connection reset by peer in connection to symfony.com:443
* Closing connection
curl: (35) Recv failure: Connection reset by peer
root@micha:~# wget --spider https://symfony.com
Spider mode enabled. Check if remote file exists.
--2024-05-01 17:09:16-- https://symfony.com/
Resolving symfony.com (symfony.com)... 2606:4700:20::ac43:4826, 2606:4700:20::681a:5fb, 2606:4700:20::681a:4fb, ...
Connecting to symfony.com (symfony.com)|2606:4700:20::ac43:4826|:443... connected.
GnuTLS: Error in the pull function.
Unable to establish SSL connection.
root@micha:~# openssl s_client -connect symfony.com:443 <<< Q
CONNECTED(00000003)
write:errno=104
---
no peer certificate available
---
No client certificate CA names sent
---
SSL handshake has read 0 bytes and written 313 bytes
Verification: OK
---
New, (NONE), Cipher is (NONE)
This TLS version forbids renegotiation.
Compression: NONE
Expansion: NONE
No ALPN negotiated
Early data was not sent
Verify return code: 0 (ok)
---
root@micha:~# openssl s_client -connect symfony.com:443 -tls1_2 <<< Q
CONNECTED(00000003)
write:errno=104
---
no peer certificate available
---
No client certificate CA names sent
---
SSL handshake has read 0 bytes and written 208 bytes
Verification: OK
---
New, (NONE), Cipher is (NONE)
Secure Renegotiation IS NOT supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
SSL-Session:
Protocol : TLSv1.2
Cipher : 0000
Session-ID:
Session-ID-ctx:
Master-Key:
PSK identity: None
PSK identity hint: None
SRP username: None
Start Time: 1714576373
Timeout : 7200 (sec)
Verify return code: 0 (ok)
Extended master secret: no
---
root@micha:~# openssl s_client -connect symfony.com:443 -debug -trace <<< Q
CONNECTED(00000003)
Sent Record
Header:
Version = TLS 1.0 (0x301)
Content Type = Handshake (22)
Length = 308
ClientHello, Length=304
client_version=0x303 (TLS 1.2)
Random:
gmt_unix_time=0x45567EF4
random_bytes (len=28): 32B39FB47DF50C03A431BE97C4C52AB034A0D49625A423C5B61F3EFD
session_id (len=32): D90F209EC84015539B5A2F8A4FB1D8E109015647315BFA8492063650F76FA259
cipher_suites (len=62)
{0x13, 0x02} TLS_AES_256_GCM_SHA384
{0x13, 0x03} TLS_CHACHA20_POLY1305_SHA256
{0x13, 0x01} TLS_AES_128_GCM_SHA256
{0xC0, 0x2C} TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384
{0xC0, 0x30} TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384
{0x00, 0x9F} TLS_DHE_RSA_WITH_AES_256_GCM_SHA384
{0xCC, 0xA9} TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256
{0xCC, 0xA8} TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256
{0xCC, 0xAA} TLS_DHE_RSA_WITH_CHACHA20_POLY1305_SHA256
{0xC0, 0x2B} TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256
{0xC0, 0x2F} TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
{0x00, 0x9E} TLS_DHE_RSA_WITH_AES_128_GCM_SHA256
{0xC0, 0x24} TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384
{0xC0, 0x28} TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384
{0x00, 0x6B} TLS_DHE_RSA_WITH_AES_256_CBC_SHA256
{0xC0, 0x23} TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256
{0xC0, 0x27} TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256
{0x00, 0x67} TLS_DHE_RSA_WITH_AES_128_CBC_SHA256
{0xC0, 0x0A} TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA
{0xC0, 0x14} TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA
{0x00, 0x39} TLS_DHE_RSA_WITH_AES_256_CBC_SHA
{0xC0, 0x09} TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA
{0xC0, 0x13} TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA
{0x00, 0x33} TLS_DHE_RSA_WITH_AES_128_CBC_SHA
{0x00, 0x9D} TLS_RSA_WITH_AES_256_GCM_SHA384
{0x00, 0x9C} TLS_RSA_WITH_AES_128_GCM_SHA256
{0x00, 0x3D} TLS_RSA_WITH_AES_256_CBC_SHA256
{0x00, 0x3C} TLS_RSA_WITH_AES_128_CBC_SHA256
{0x00, 0x35} TLS_RSA_WITH_AES_256_CBC_SHA
{0x00, 0x2F} TLS_RSA_WITH_AES_128_CBC_SHA
{0x00, 0xFF} TLS_EMPTY_RENEGOTIATION_INFO_SCSV
compression_methods (len=1)
No Compression (0x00)
extensions, length = 169
extension_type=server_name(0), length=16
0000 - 00 0e 00 00 0b 73 79 6d-66 6f 6e 79 2e 63 6f .....symfony.co
000f - 6d m
extension_type=ec_point_formats(11), length=4
uncompressed (0)
ansiX962_compressed_prime (1)
ansiX962_compressed_char2 (2)
extension_type=supported_groups(10), length=22
ecdh_x25519 (29)
secp256r1 (P-256) (23)
ecdh_x448 (30)
secp521r1 (P-521) (25)
secp384r1 (P-384) (24)
ffdhe2048 (256)
ffdhe3072 (257)
ffdhe4096 (258)
ffdhe6144 (259)
ffdhe8192 (260)
extension_type=session_ticket(35), length=0
extension_type=encrypt_then_mac(22), length=0
extension_type=extended_master_secret(23), length=0
extension_type=signature_algorithms(13), length=42
ecdsa_secp256r1_sha256 (0x0403)
ecdsa_secp384r1_sha384 (0x0503)
ecdsa_secp521r1_sha512 (0x0603)
ed25519 (0x0807)
ed448 (0x0808)
rsa_pss_pss_sha256 (0x0809)
rsa_pss_pss_sha384 (0x080a)
rsa_pss_pss_sha512 (0x080b)
rsa_pss_rsae_sha256 (0x0804)
rsa_pss_rsae_sha384 (0x0805)
rsa_pss_rsae_sha512 (0x0806)
rsa_pkcs1_sha256 (0x0401)
rsa_pkcs1_sha384 (0x0501)
rsa_pkcs1_sha512 (0x0601)
ecdsa_sha224 (0x0303)
rsa_pkcs1_sha224 (0x0301)
dsa_sha224 (0x0302)
dsa_sha256 (0x0402)
dsa_sha384 (0x0502)
dsa_sha512 (0x0602)
extension_type=supported_versions(43), length=5
TLS 1.3 (772)
TLS 1.2 (771)
extension_type=psk_key_exchange_modes(45), length=2
psk_dhe_ke (1)
extension_type=key_share(51), length=38
NamedGroup: ecdh_x25519 (29)
key_exchange: (len=32): B3BFFF739DE8D340A6F4F4ED3E8FDD3C2C35A7342F2E119094B0BBBBBE9F8970
write to 0x20f2bd8 [0x21b5910] (313 bytes => 313 (0x139))
0000 - 16 03 01 01 34 01 00 01-30 03 03 45 56 7e f4 32 ....4...0..EV~.2
0010 - b3 9f b4 7d f5 0c 03 a4-31 be 97 c4 c5 2a b0 34 ...}....1....*.4
0020 - a0 d4 96 25 a4 23 c5 b6-1f 3e fd 20 d9 0f 20 9e ...%.#...>. .. .
0030 - c8 40 15 53 9b 5a 2f 8a-4f b1 d8 e1 09 01 56 47 [email protected]/.O.....VG
0040 - 31 5b fa 84 92 06 36 50-f7 6f a2 59 00 3e 13 02 1[....6P.o.Y.>..
0050 - 13 03 13 01 c0 2c c0 30-00 9f cc a9 cc a8 cc aa .....,.0........
0060 - c0 2b c0 2f 00 9e c0 24-c0 28 00 6b c0 23 c0 27 .+./...$.(.k.#.'
0070 - 00 67 c0 0a c0 14 00 39-c0 09 c0 13 00 33 00 9d .g.....9.....3..
0080 - 00 9c 00 3d 00 3c 00 35-00 2f 00 ff 01 00 00 a9 ...=.<.5./......
0090 - 00 00 00 10 00 0e 00 00-0b 73 79 6d 66 6f 6e 79 .........symfony
00a0 - 2e 63 6f 6d 00 0b 00 04-03 00 01 02 00 0a 00 16 .com............
00b0 - 00 14 00 1d 00 17 00 1e-00 19 00 18 01 00 01 01 ................
00c0 - 01 02 01 03 01 04 00 23-00 00 00 16 00 00 00 17 .......#........
00d0 - 00 00 00 0d 00 2a 00 28-04 03 05 03 06 03 08 07 .....*.(........
00e0 - 08 08 08 09 08 0a 08 0b-08 04 08 05 08 06 04 01 ................
00f0 - 05 01 06 01 03 03 03 01-03 02 04 02 05 02 06 02 ................
0100 - 00 2b 00 05 04 03 04 03-03 00 2d 00 02 01 01 00 .+........-.....
0110 - 33 00 26 00 24 00 1d 00-20 b3 bf ff 73 9d e8 d3 3.&.$... ...s...
0120 - 40 a6 f4 f4 ed 3e 8f dd-3c 2c 35 a7 34 2f 2e 11 @....>..<,5.4/..
0130 - 90 94 b0 bb bb be 9f 89-70 ........p
read from 0x20f2bd8 [0x21ac6eb] (5 bytes => -1)
write:errno=104
---
no peer certificate available
---
No client certificate CA names sent
---
SSL handshake has read 0 bytes and written 313 bytes
Verification: OK
---
New, (NONE), Cipher is (NONE)
This TLS version forbids renegotiation.
Compression: NONE
Expansion: NONE
No ALPN negotiated
Early data was not sent
Verify return code: 0 (ok)
---
read from 0x20f2bd8 [0x20d9b30] (8192 bytes => 0)
And from Windows:
C:\Users\Micha>curl -I https://symfony.com
curl: (35) Recv failure: Connection was reset
tcpdump
17:35:24.225636 IP6 my.host.45736 > 2606:4700:20::681a:5f3.https: Flags [S], seq 206561763, win 32936, options [mss 1432,sackOK,TS val 47603026 ecr 0,nop,wscale 7], length 0
17:35:24.250033 IP6 2606:4700:20::681a:5f3.https > my.host.45736: Flags [S.], seq 2974905604, ack 206561764, win 65535, options [mss 1360,sackOK,TS val 2728680421 ecr 47603026,nop,wscale 13], length 0
17:35:24.250286 IP6 my.host.45736 > 2606:4700:20::681a:5f3.https: Flags [.], ack 1, win 258, options [nop,nop,TS val 47603050 ecr 2728680421], length 0
17:35:24.266882 IP6 my.host.45736 > 2606:4700:20::681a:5f3.https: Flags [P.], seq 1:518, ack 1, win 258, options [nop,nop,TS val 47603067 ecr 2728680421], length 517
17:35:24.289647 IP6 2606:4700:20::681a:5f3.https > my.host.45736: Flags [R], seq 2974905605, win 0, length 0
I face the same with a regular web browser when accessing websites behind Cloudflare. At first I was not associating it with this issue, but in cases of affected websites, the very first access fails, and my browser then immediately retries (in case with plain HTTP, if no HSTS was stored) so that the website usually opens shortly after, or after a manual reload. When reloading an affected website, I then by times see something like this in browser tools network tab:
symfony.com (canceled) document Other 0 B 38 ms
- The error happens independent of host machine/OS/version, TLS library/version, TLS protocol version, client/version.
- The error happens independent of the exact Cloudflare Edge responding (at my location). The same Edge/IP addresses are sometimes sending a functional response, sometimes cause the error.
- The error happens also independent of whether the content exists, is cached, handled by a worker or anyhow otherwise treated. I happens really on all possible URLs, basically on a level below that, as long as it is a TLS connection through Cloudflare.
- The error happens ONLY when accessing via IPv6, while I was not able to ever replicate it with IPv4. This contradicts other reports, where the same TLS errors and symptoms are faced with IPv4 as well. Probably has only an indirect effect.
- The error does not happen on all, but many websites behind Cloudflare.
- The error happens intermittently in roughly 50% of cases when running the very same command multiple times one after another, while it changes on a daily/hourly basis, with times of 100% success and times of much worse error rate.
- No firewall is involved, also Cloudflare does not show any WAF event.
- The issue does not affect only me, but some of our visitors reported the same. There are other reports here on the forum, just to list two cases (while there are many more when searching for the particular error messages based on GnuTLS vs OpenSSL):
Basically, especially the last openssl s_client trace+debug output and tcpdump show, that the client sends a handshake request (which is exactly the same on successful connections), but gets an empty response from the peer.
I hope that I could make clear, that while it is affecting a small number of users only, it is a Cloudflare bug which needs investigation. Since the individual platform and client is completely irrelevant, it must have something to do with the network/router/ISP/route (while network, router and ISP do not have issues with any other website, or the same when bypassing Cloudflare, where possible).
I can give details regarding this and can run more tests ASAP. Probably the two pinged guys, in case they are still affected by the issue, can verify that the points I listed above are the same for them (aside of “IPv6-only”).
: 
. So one can see the successful request was captured, the keys logged successfully, and