tl;dr: Let Let’s Encrypt bot bypass Always Use HTTPS so certs can be renewed automatically, so that users don’t need to temporarily turn it off, un-proxy domains or pause Cloudflare.
The Always Use HTTPS is a feature Cloudflare launched 3 years ago to help users make sure their website is always redirected to HTTPS, as part of its broader effort to popularize the use of HTTPS.
Let’s Encrypt is a free SSL certificate issuer. It’s bot for certificate renewal needs to find the server via HTTP, and it stops midway in case there’s a redirect that is performed by a proxy (redirects from HTTP to HTTPS at the origin do not prevent LE cert renewal).
I’m suggesting that Cloudflare set a bypass for the LE bot under its Always Use HTTPS feature.
Website owners with their sites on shared hosting often do not have the capability (and many, like me, do not have the technical skills) to circumvent this situation. They depend on the bot performing as set by the hosting company.
When users come to the Cloudflare Community asking why their certs are not renewing, the solution proposed is often to either pause Cloudflare (bad), un-proxy the relevant domain (still bad), or turn off Always Use HTTPS (not the ideal) for the time it takes for the bot to perform the renewal.
The problem is that Cloudflare should not be paused, and domains should not be un-proxied, and the idea of toggling AUH on and back off takes the automatic from the process, making it error prone. A user may toggle AUH temporarily for a day or a few hours, but that may become several days in case one forgets
Cloudflare champions the idea that HTTPS should always be used. For that reason, it created Universal SSL, then Flexible to allow for site without SSL at the origin to benefit from the free SSL, then solutions for mixed content etc.
I believe it would make sense to allow Let’s Encrypt bot bypass AUH.
The alternative would be page rules. But for page rules to work, the user would necessarily have to turn AUH off at the dashboard, and on for the zone with an extra page rule to except the LE bot. Any misstep on the configuration of page rules would leave their sites or subdomains vulnerable (not to mention eventual SEO penalties) . Also, one may need to create many page rules depending on how many subdomains the zone has. And with many page rules, the user has to keep up with a kind of maintenance nightmare.