DNS-over-TLS broken with Knot Resolver since today

Zero Trust 1.1.1.1
1

Since a couple of hours ago, more precisely 13 January 2020 around 14:30 GMT, all my Knot Resolver instances, running 3.2.1 on Debian, started failing to forward DNS requests to 1.1.1.1 over TLS.

Configuration is like this:

policy.add(policy.all(policy.TLS_FORWARD({
         { '1.1.1.1', hostname='cloudflare-dns.com', ca_file='/etc/ssl/certs/ca-certificates.crt' },
})))

Logs say this:

kresd[8129]: [tls_client] TLS handshake with 1.1.1.1#00853 has completed
kresd[8129]: [tls_client] TLS session has not resumed
kresd[8129]: [gnutls] (5) REC[0x55868f25e580]: Preparing Packet Application Data(23) with length: 41 and min pad: 0
kresd[8129]: [gnutls] (5) REC[0x55868f25e580]: Sent Packet[1] Application Data(23) in epoch 2 and length: 63
kresd[8129]: [gnutls] (3) ASSERT: ../../lib/buffers.c[_gnutls_io_read_buffered]:589
kresd[8129]: [gnutls] (3) ASSERT: ../../lib/record.c[_gnutls_recv_int]:1775
kresd[8129]: [gnutls] (5) REC[0x55868f25e580]: SSL 3.3 Application Data packet received. Epoch 2, length: 445
kresd[8129]: [gnutls] (5) REC[0x55868f25e580]: Expected Packet Application Data(23)
kresd[8129]: [gnutls] (5) REC[0x55868f25e580]: Received Packet Application Data(23) with length: 445
kresd[8129]: [gnutls] (5) REC[0x55868f25e580]: Decrypted Packet[0] Handshake(22) with length: 428
kresd[8129]: [gnutls] (3) ASSERT: ../../lib/buffers.c[get_last_packet]:1168
kresd[8129]: [gnutls] (4) HSK[0x55868f25e580]: NEW SESSION TICKET (4) was received. Length 210[424], frag offset 0, frag length: 210, sequence: 0
kresd[8129]: [gnutls] (3) ASSERT: ../../lib/buffers.c[_gnutls_handshake_io_recv_int]:1429
kresd[8129]: [gnutls] (4) HSK[0x55868f25e580]: parsing session ticket message
kresd[8129]: [gnutls] (3) ASSERT: ../../lib/buffers.c[get_last_packet]:1168
kresd[8129]: [gnutls] (4) HSK[0x55868f25e580]: NEW SESSION TICKET (4) was received. Length 210[210], frag offset 0, frag length: 210, sequence: 0
kresd[8129]: [gnutls] (3) ASSERT: ../../lib/buffers.c[_gnutls_handshake_io_recv_int]:1429
kresd[8129]: [gnutls] (4) HSK[0x55868f25e580]: parsing session ticket message
kresd[8129]: [gnutls] (3) ASSERT: ../../lib/record.c[_gnutls_recv_in_buffers]:1577
kresd[8129]: [gnutls] (3) ASSERT: ../../lib/record.c[_gnutls_recv_int]:1775
kresd[8129]: [io] => connection to '1.1.1.1#00853': error processing TLS data, close

So this started happening on 4 different Knot Resolver instances, with similar configuration, on different ISPs, all at the same time. They can perfectly TLS_FORWARD to other DNS servers though. It looks like Cloudflare changed something which triggers this problem?

2

Hi Fre,

A recent release re-enabled TLS 1.3 including session resumption support for DoT. It looks like an implementation detail (DNS application data preceded by New Session Ticket in the same TLS record with TLS 1.3) results in Knot Resolver getting upset. This problem has been addressed by knot-resolver 4.2.0 (see [tls_client] session resumption doesn't work when server sends session ticket along with other data (#489) · Issues · Knot projects / Knot Resolver · GitLab and Knot Resolver 4.2.0 released – Knot Resolver).

As an immediate solution, you can try to upgrade your client to use the newer upstream packages (Download – Knot Resolver). Requesting Debian to cherry-pick the patch is another option.
Disabling session resumption unfortunately does not seem to be a configurable option in Knot Resolver.

As hack, you can indirectly disable session ticket support by patching the priority string in the kresd 3.2.1 binary to use the “PFS” class of ciphers instead of “NORMAL”. Do keep the number of characters the same, there are three spaces before “PFS”:

$ strings /usr/sbin/kresd | grep NORMAL
NORMAL:-VERS-TLS1.0:-VERS-TLS1.1:-VERS-SSL3.0:-ARCFOUR-128:-COMP-ALL:+COMP-NULL
$ sudo sed 's/NORMAL/   PFS/' -i /usr/sbin/kresd

Documentation for PFS is available at Priority Strings (GnuTLS 3.8.2)

%NO_TICKETS will prevent the advertizing of the TLS session ticket extension. This is implied by the PFS keyword.

The relevant source code that is being patched can be found here: daemon/tls.c · v3.2.1 · Knot projects / Knot Resolver · GitLab

We’ll look into ways to restore compatibility with broken clients, but some trade off will have to be made.

6 Likes
3

Thanks. Why didn’t I discover this bug ticket myself :angry:

1 Like