Split brain DNS w/ internal scope of RFC1918 class B network

We use split brain DNS with an External scope for both our public /20 as well as the random external IPs provided by google in our google cloud infrastructure.

We have many internal virtual machines that only have RFC1918 24BitBlock Private IPs. These machines are made accessible to the outside world using NAT on our firewall. An internal machine cannot get to the external interface of the firewall for the NAT to occur, so we need to resolve the same domain name to a private address for internal, VPN, and WAN users, and to a pubic address for those that are off campus. Easy peasy right?

Unfortunately we already have users come to campus with statically configured, or aggressively cached DNS, or WAN users configured to get their DHCP Settings from their ISP, who are unable to get to our internal servers.

We are considering moving our DNS to Cloudflare.

I have some concerns about exacerbating this issue, and managing private IPs and a split brain architecture through Cloudflare.

Has anyone found an elegant solution to this issue, Cloudflare or not?

Does anyone have experience managing private IP address space at Cloudflare?

Can anyone with more experience offer any suggestions?

Thanks in advance for your responses.

Alias

I know split brain DNS comes up from time to time here. Give it a search to see if that helps, and maybe someone here has some suggestions as well. @cs-cf might know a thing or two about DNS as well.

Ugh… this is (as you clearly already know) your biggest problem right here. :facepalm:

Understandable. Not having a totally complete picture of your environment I can offer some suggestions based on previous experiences. You may want to follow-up with our Enterprise Sales team as well and a dedicated solutions engineer can ask further questions. But for the moment based on what you’ve told me my suggestions would be (not in priority order):

Suggestion 1.

A. User an ‘internal’ DNS server which supports EDNS-Client-Subnet. For your internal IP address range (and the IP address(es) of the machine(s) internal requests would be going to Cloudflare set the IP address to the internal address of the machines. For all other IP addresses return the ‘external’ IP address.

B. Use Cloudflare’s DNS Firewall product to place your ‘internal DNS’ Server behind Cloudflare’s DNS proxy which supports EDNS-Client-Subnet and change your nameservers to point to that so users using external resolvers (except ironically our 1.1.1.1 service) on the internal network should get the right IP address when querying… aggressively caching resolvers and resolvers which don’t support ECS excluded.

Suggestion 2:

A. Use Cloudflare for external DNS AND Proxy.

B. Proxy as many of those internal hostnames as is reasonable/possible through Cloudflare… both standard proxy for web apps and Spectrum for non-web applications so that users resolving to the ‘external IP’ will really be resolving to Cloudflare’s proxy which will eliminate the problem of an internal machine trying to hit the external interface of the firewall.

C. Look at Cloudflare Access as well to potentially add another layer of security around sites you might not be exposing today (or to better protect ones you are).

1 Like

Only ever done this at smaller places but the easiest thing I’ve found is something that isn’t going to be Cloudflare specific but it’s simple and works:

  1. External DNS (Cloudflare) hosts the public IP of those internal servers which are behind your NAT to maintain external connectivity
  2. An internal DNS hosts the RFC1918 names for these hosts and uses an upstream resolver for anything not defined locally (this could be 1.1.1.1, say)
  3. Implement a DNAT rule on your network edge for all outbound DNS traffic (UDP/TCP53 in the pre-encryption days) which routes all DNS requests to your internal server (except for requests from that server itself, naturally).

It won’t solve caching issues, but with Cloudflare as your authoriative DNS you can set the TTLs to be low-low-low anyway. Should cover everything else though and it’s easy. One internal service and a couple of firewall rules. The latter actually is useful anyway for keeping DNS all under control in-house and looks like the only thing you’re missing??

So is there a way to delegate the internal scope of our DNS from Cloudflare to us or do we just rely on people using the on-prem DNS servers for resolution to avoid them trying to go find the root servers and so on for RFC1918 addresses?

… and the DNAT rule forces all internal requests to go through the on prem dns servers (answering my question above) and all external queries are handled through recursion.

That sounds really great, I like simple. So the Cloudflare DNS servers don’t even know about the on prem domains… no axfr from Cloudflare to the on prem servers…

while we’re at it, I don’t suppose you know of any usable open source web interfaces for managing DNS and DHCP servers for those allergic to ascii do you ?

we used probind ages ago, but it was still to complicated for most to use

Thanks so much for responding.

Well @cs-cf is staff and he’ll have forgotten more about DNS than I’ll ever know but in my solution you’re relying on the people onsite or connected through your VPN resolving via your inhouse DNS server - and the DNAT rule absolutely forces that. The only way they wouldn’t is if the record was cached on their device but if you set low TTLs that shouldn’t be an issue.

You’re right the Cloudflare won’t know about nor care about the onsite DNS. They won’t host any RFC1918 addresses just the stuff you need out there for public access (which could of course be the same record names as you also use inhouse if you need internal and external access).

As for your inhouse server, most DNS servers should work - starting as simple as the lowly soho-style dnsmasq right up to BIND and other standard servers but they’re not-GUI based. I’m sure there must be some management tools out there but its not something I’ve ever looked into, sorry.

Ya, me neither, I’m much more comfortable in the *nix world and I’ve bonded with Bind many times, so if it supports DNAT, that sounds like a great solution. Also, I’m looking for something for other people to serve themselves so that doesn’t become my entire job. I did a quick google search and found something that looks pretty good…NIPAP, built by network engineers, in python, open source, with a command-line interface so I can still get my sed and awk on.

I’m hoping they don’t bork the config files too much in order to make it easier to update programmatically.

We’re currently using Bluecat’s IPAM solution and the config files are a bit of a mess, though this is the only time I’ve ever had to look at them.

Also looking for a modern wireless authentication system that will tie in with the DHCP server.

Anyhow thanks again for all of your help, improved my spirits for certain.

Cian

This topic was automatically closed after 31 days. New replies are no longer allowed.