Cloudflare’s IP addresses rarely change. I have my servers set to block everything except requests from Cloudflare IP addresses.
As you suspect, by checking hostnames on your server, you’re safe. I’m pretty sure Cloudflare isn’t going to bother your server with incorrect domain name requests.