I am afraid that is rather a question for StackOverflow.
You should not use a self-signed certificate however, as that wouldn’t be secure in either context. Use Lets Encrypt instead. What you need to do is get that service up and running without Cloudflare and once it works, you can add Cloudflare to it.
You could also get a Cloudflare Origin certificate instead, but that could be an issue during testing without Cloudflare as the connection would be still untrusted.
This error disappeared when I changed DNS mode from “proxied” to “dns only”. Does this disable cloudflare’s anti-ddos protection? If yes, then how to deal with this? If no, then what does this switcher change??