Disabled CNAME flattening, cloudflare still responds with cached records

Hi,

I have a CNAME record in Cloudflare DNS as follows, to delegate acme DNS requests to an external DNS:


CNAME     _acme-challenge     MYDOMAIN.com.acme.nhcloud.MYDOMAIN.com     [DNS only] TTL 1 min


This is used for LetsEncrypt DNS acme validation. This has started to fail recently. I figured this might be due to CNAME flattening, which I had enabled for all hosts in the zone. I've now switch to 'Flatten CNAME at root'.

Yet, more than 8 hours after disabling CNAME flattening, cloudflare DNS is still responding with cached TXT records:


dig _acme-challenge.MYDOMAIN.com txt @nile.ns.cloudflare.com

; <<>> DiG 9.16.1-Ubuntu <<>> _acme-challenge.MYDOMAIN.com txt @nile.ns.cloudflare.com

;; global options: +cmd

;; Got answer:

;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 42793

;; flags: qr aa rd; QUERY: 1, ANSWER: 4, AUTHORITY: 0, ADDITIONAL: 1

;; WARNING: recursion requested but not available

;; OPT PSEUDOSECTION:

; EDNS: version: 0, flags:; udp: 1232

;; QUESTION SECTION:

;_acme-challenge.MYDOMAIN.com. IN TXT

;; ANSWER SECTION:

_acme-challenge.MYDOMAIN.com. 300 IN TXT "gBj9M_7HM (redacted) 1kp_wF8SJHaAYFk2GAsSW60UM"

_acme-challenge.MYDOMAIN.com. 300 IN TXT "fCkX6SLIL (redacted) PkS3FxaGxuJFsk"

_acme-challenge.MYDOMAIN.com. 300 IN TXT "r3fu7dceO (redacted) rtO5_tbB_seawtAwnCsNeX1c"

_acme-challenge.MYDOMAIN.com. 300 IN TXT "Fcr1aYboor (redacted) H-g4J1YYmVCYtVvvto"

;; Query time: 8 msec

;; SERVER: 2a06:98c1:50::ac40:21d6#53(2a06:98c1:50::ac40:21d6)

;; WHEN: Fri Jun 24 09:18:47 CEST 2022

;; MSG SIZE rcvd: 278


The other weird thing is I'm seeing multiple TXT records, while my actual DNS (acme-dns-certbot) is only responding with a single one. It seems your CNAME flattening is eternally caching my TXT records.

I've tried renaming and deleting the _acme-challenge entry, but as soon as I add it again, I'm served the cached data once again. So I'm stuck and can't renew my letsencrypt certificate.

Also I would expect this to work, even with CNAME flattening enabled. The certbot DNS is responding with 60 seconds TTL.

This looks like a bug. Could someone contact me to resolve this issue?

Thanks from a Cloudflare Pro customer

and sorry for the weird formatting, for some reason your forum is replacing every occurrence of the word CNAME with a link, and then erroring because I have more than 4 hyperlinks in my post :confounded:

CNAME flattening has nothing to do with caching records - CNAME flattening turns CNAMEs into IP addresses.

What are you saying is ‘cached’? If you have a CNAME and lookup the TXT records for that CNAME, you will get the TXT records from the target your CNAME is pointing to.

What are you saying is ‘cached’? If you have a CNME and lookup the TXT records for that CNME, you will get the TXT records from the target your CN*ME is pointing to.

Indeed, that’s what I expect. But Cloudflare is still returning TXT records.

$ dig _acme-challenge.MYDOMAIN.com txt @nile.ns.cloudflare.com

; <<>> DiG 9.18.1-1ubuntu1.1-Ubuntu <<>> _acme-challenge.MYDOMAIN.com txt @nile.ns.cloudflare.com
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 11642
;; flags: qr aa rd; QUERY: 1, ANSWER: 4, AUTHORITY: 0, ADDITIONAL: 1
;; WARNING: recursion requested but not available

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 1232
;; QUESTION SECTION:
;_acme-challenge.MYDOMAIN.com.	IN	TXT

;; ANSWER SECTION:
_acme-challenge.MYDOMAIN.com. 300	IN	TXT	"gBj9M_HM1JRlQ (redacted) JHaAGAsSW60UM"
_acme-challenge.MYDOMAIN.com. 300	IN	TXT	"fCkX6SLILF2x_XU (redacted) BQBCNxuJFsk"
_acme-challenge.MYDOMAIN.com. 300	IN	TXT	"r3fu7dceO (redacted) tbB_se1yQqtAwnCsNeX1c"
_acme-challenge.MYDOMAIN.com. 300	IN	TXT	"Fcr1aYboorH (redacted) 1YYmVCYtVvvto"

;; Query time: 12 msec
;; SERVER: 173.245.59.214#53(nile.ns.cloudflare.com) (UDP)
;; WHEN: Fri Jun 24 13:32:06 CEST 2022
;; MSG SIZE  rcvd: 278

$ dig _acme-challenge.MYDOMAIN.com cname @nile.ns.cloudflare.com

; <<>> DiG 9.18.1-1ubuntu1.1-Ubuntu <<>> _acme-challenge.MYDOMAIN.com cname @nile.ns.cloudflare.com
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 38991
;; flags: qr aa rd; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1
;; WARNING: recursion requested but not available

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 1232
;; QUESTION SECTION:
;_acme-challenge.MYDOMAIN.com.	IN	CNAME

;; ANSWER SECTION:
_acme-challenge.MYDOMAIN.com. 60	IN	CNAME	MYDOMAIN.com.acme.nhcloud.MYDOMAIN.com.

;; Query time: 12 msec
;; SERVER: 172.64.33.214#53(nile.ns.cloudflare.com) (UDP)
;; WHEN: Fri Jun 24 13:32:15 CEST 2022
;; MSG SIZE  rcvd: 91

$ 

Letsencrypt is asking for TXT records and gets those stale responses directly from Cloudflare. Cloudflare should instead return the CN*ME response.

I don’t want to post my domain name here in a public forum, can someone from CF contact me?

I can’t really offer any other advice without knowing the domain.

Open a support ticket.

How do I open a support ticket?

‘Technical Problem’ just points me here.

Apparently I can email [email protected]
That was well hidden.

Out of curiousity, have you tried running the dig command with a recursive server like 1.1.1.1?

dig _acme-challenge.domain.com TXT @1.1.1.1

Yours are asking nile.ns.cloudflare.com which is an authoriative nameserver, not a recursive one - I’m not entirely sure if that’ll ever follow the CNAME chain, I’d assume you’re just asking Cloudflare for their TXT records.

dig _acme-challenge.domain.com TXT @1.1.1.1

This returns the same (faulty) TXT records (otherwise I assume letsencrypt would work as well).

As I said I had CNAME flattening enabled until yesterday (24h ago). Those TXT records must be at least 1 week old (and they have a TTL of 1 minute).

Also I would expect CNAME flattening to work transparently and always return fresh data (after TTL expired)

It does - but I don’t think TXT records are within the realm of CNAME flattening.

CNAME flattening will turn your CNAME into IP addresses - if it also populates the other record types such as TXT from the original CNAME isn’t something that I’m aware of and also isn’t documented.

If you’ve already contacted support then they should be able to give you a definitive answer.

1 Like

Of course not, what should it flatten :slight_smile:

I mean in the context of if you had a CNAME for foo going to foo.example.com then a TXT lookup for foo would return the TXT records from foo.example.com

With CNAME flattening, I don’t know if the TXT lookup for foo returns the ones on Cloudflare (since it’ll accept records alongside a CNAME) or the ones from foo.example.com.

Sure, of course, that’s what DNS does. I think there’s some confusion what flattening is :slight_smile:

A TXT record is a TXT record and resolving it will always return its content.

It could proxy the TXT records. There’s not much benefit, but why not? And apparently that’s what they are doing, otherwise I don’t see where those TXT results are coming from.

I’ve opened a support ticket, will let you know.

You can’t proxy TXT records.

Equally, your support ticket is unlikely to get an answer, unless you are on a paid plan.

Anyhow, @KianNH’s response above is spot on.

That’s the issue, I don’t want my TXT records to be proxied. And I am on the Pro plan.

No CNAME flattening:

;; QUESTION SECTION:
;protonmail._domainkey.kian.org.uk. IN	TXT

;; ANSWER SECTION:
protonmail._domainkey.kian.org.uk. 300 IN CNAME	protonmail.domainkey.d765ws4plgnyzhuqqyqmnohwl4jfhs6fpp6rirnozmlua5dytl22q.domains.proton.ch.
protonmail.domainkey.d765ws4plgnyzhuqqyqmnohwl4jfhs6fpp6rirnozmlua5dytl22q.domains.proton.ch. 0	IN TXT "v=DKIM1;k=rsa;p=MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAu8hHZ2FVBjQcGuDg7BlCcZ/c7CqICnwEiq5QFmSxO/AuXJuZWHuUcuKVXWlcuNkvGd0cAo7mS84riG1D/3k86eus4h8UC13Kk5OAkveGZyUhxgtAgpX6LjoLy88rieTUztLqLXAqyIMh4L6uflCP9S/tfHu+rr6+JqsD14Z59ggfydSPJf0sL8O+Sdj7g9dRfYI" "cvJKTnBGqlIvZyTYOtEAw4P2lPS+RY7cvgbUMobgeldCGSUcfgt2OCvEw+WJUoE8sN0wO/H+F1VVybfAFhz8fB2lq3Yf07v2ydBXhqYo0OZPqTlwFCB9Q817bgMAgtQ6iAb84rMxksn2Q1IzyUQIDAQAB;"

The lookup follows the CNAME chain to find the TXT records on protonmail.domainkey.d765ws4plgnyzhuqqyqmnohwl4jfhs6fpp6rirnozmlua5dytl22q.domains.proton.ch.

With CNAME flattening:

;; QUESTION SECTION:
;protonmail._domainkey.kian.org.uk. IN	TXT

;; ANSWER SECTION:
protonmail._domainkey.kian.org.uk. 264 IN TXT	"v=DKIM1;k=rsa;p=MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAu8hHZ2FVBjQcGuDg7BlCcZ/c7CqICnwEiq5QFmSxO/AuXJuZWHuUcuKVXWlcuNkvGd0cAo7mS84riG1D/3k86eus4h8UC13Kk5OAkveGZyUhxgtAgpX6LjoLy88rieTUztLqLXAqyIMh4L6uflCP9S/tfHu+rr6+JqsD14Z59ggfydSPJf0sL8O+Sdj7g9dRfYI" "cvJKTnBGqlIvZyTYOtEAw4P2lPS+RY7cvgbUMobgeldCGSUcfgt2OCvEw+WJUoE8sN0wO/H+F1VVybfAFhz8fB2lq3Yf07v2ydBXhqYo0OZPqTlwFCB9Q817bgMAgtQ6iAb84rMxksn2Q1IzyUQIDAQAB;

Cloudflare does carry over other record types which was what I was wondering about.

So CNAME flattening or not, the TXT records of your CNAME target would be returned.

And your TXT won’t be proxied.

All right, on a Pro plan you should get a response, but @KianNH already addressed it, so you can close the ticket.

1 Like

With all respect @sandro, but my problem is not solved until letsencrypt sees the correct TXT records.