Let Let's Encrypt bot bypass Always Use HTTPS

Feedback Feature Request Submitting & Feedback
,
1

tl;dr: Let Let’s Encrypt bot bypass Always Use HTTPS so certs can be renewed automatically, so that users don’t need to temporarily turn it off, un-proxy domains or pause Cloudflare.

The Always Use HTTPS is a feature Cloudflare launched 3 years ago to help users make sure their website is always redirected to HTTPS, as part of its broader effort to popularize the use of HTTPS.

Let’s Encrypt is a free SSL certificate issuer. It’s bot for certificate renewal needs to find the server via HTTP, and it stops midway in case there’s a redirect that is performed by a proxy (redirects from HTTP to HTTPS at the origin do not prevent LE cert renewal).

I’m suggesting that Cloudflare set a bypass for the LE bot under its Always Use HTTPS feature.

Website owners with their sites on shared hosting often do not have the capability (and many, like me, do not have the technical skills) to circumvent this situation. They depend on the bot performing as set by the hosting company.

When users come to the Cloudflare Community asking why their certs are not renewing, the solution proposed is often to either pause Cloudflare (bad), un-proxy the relevant domain (still bad), or turn off Always Use HTTPS (not the ideal) for the time it takes for the bot to perform the renewal.

The problem is that Cloudflare should not be paused, and domains should not be un-proxied, and the idea of toggling AUH on and back off takes the automatic from the process, making it error prone. A user may toggle AUH temporarily for a day or a few hours, but that may become several days in case one forgets

Cloudflare champions the idea that HTTPS should always be used. For that reason, it created Universal SSL, then Flexible to allow for site without SSL at the origin to benefit from the free SSL, then solutions for mixed content etc.

I believe it would make sense to allow Let’s Encrypt bot bypass AUH.

The alternative would be page rules. But for page rules to work, the user would necessarily have to turn AUH off at the dashboard, and on for the zone with an extra page rule to except the LE bot. Any misstep on the configuration of page rules would leave their sites or subdomains vulnerable (not to mention eventual SEO penalties) . Also, one may need to create many page rules depending on how many subdomains the zone has. And with many page rules, the user has to keep up with a kind of maintenance nightmare.

2

Very good IDEA. I tried to renew mine yesterday and failed.

3

I ran a simple test with page rules, via turning off Always Use HTTPS at the SSL page, then create a page rule for */.well-known/acme-challenge/* with SSL to off, followed by a second page rule for * that turns Always Use HTTPS back on.

This of course is not the ideal solution, but it shouldn’t require any manual intervention over time.

Agreed that this should be something more of a checkbox/actual page rule item.

4

This also impeds domain and mail service providers who need to generate SSL certs for their customers.

Applying a default exception for /.well-known/acme-challenge/ seems like a valid solution. Many people have lost hours on this for years now.

There are possibly other legitimate use cases where HTTP is required. For instance Thunderbird’s autoconfig checks an HTTP URL. If that URL redirects to HTTPS it usually works too, but if that HTTPS URL dynamically generates a cert the first time it’s accessed the request may take some time to respond and in our experience Thunderbird’s autoconfig can fail because of that. While it doesn’t with just HTTP and simply doesn’t need HTTPS.

I get that Cloudflare wants to push HTTPS, but breaking functionality is not the way to go.

5

That should be unnecessary, certainly not by default.

Let’s Encrypt will follow up to 10 redirects looking for the challenge response. In my own testing this did not create any issue.

The only issue I have ever encountered is where I set Full (Strict) before getting the first certificate on the origin.

Some users may also encounter an issue where their hosting provider has separate directories for httpdocs and ssldocs (or similar naming conventions), and that the ACME client is putting the challenge in the wrong directory. Plesk for example has a “Use a single directory for housing SSL and non-SSL content” setting which would be helpful in that kind of situation.

There is no situation where Cloudflare will force you to use HTTPS on your own websites. “Always Use HTTPS” is a setting that any user can configure (or not), and if you need HTTP access to certain resources there are various ways to deal with that.