SSL causing website to not load at all

:flushed:

try this set TLS minimum to TLS v1.2 and re-enable HTTP/2

edit also if using Apache origin web server make sure h2c isn’t set in your protocol supported list

Protocols h2 h2c http/1.1

so maybe

Protocols h2 http/1.1

Can we please go back to unencrypted HTTP 1.0 (who cares about host headers)? Things were so much easier back then :wink:

</rant>

@cloonan, any idea why that broke on Safari, but only for the proxied page and not cdn-cgi?

This was bothering me so I dug into old support tickets and found a comment on a similar ticket to head of line blocking that pointed to this article, The Full Picture on HTTP/2 and HOL Blocking - Salesforce Engineering Blog.

This line makes me think it’s latency over the mobile network, “…unexpectedly large variations in latency or a loss affecting the segment at the TCP head of line makes HTTP/2 more likely to hit HOLB, and its impact is larger as well. Basically, the receiver stands idle till the head is recovered, while all the subsequent segments are held by the TCP stack. This means that an image might be downloaded successfully and still not show up due to HOLB.”

1 Like

yeah i’d purge cf cache globally and verify Cloudflare proxied requests are not serving Upgrade header in your requests

also ensure you restarted apache after making the change to remove h2c

Purged, and yes, I’ve restarted apache a dozen times. :slight_smile:

When I check the headers of https://test.*, it’s showing me an upgrade header: “upgrade: http/1.”

During testing this afternoon, I found out that I can replicate the issue with cURL!

With http2 enabled:

laura@york:/etc/apache2$ curl -v --http2 https://test.skylon.com

  • Rebuilt URL to: https://test.skylon.com/
  • Trying 104.26.5.63…
  • TCP_NODELAY set
  • Connected to test.skylon.com (104.26.5.63) port 443 (#0)
  • ALPN, offering h2
  • ALPN, offering http/1.1
  • successfully set certificate verify locations:
  • CAfile: /etc/ssl/certs/ca-certificates.crt
    CApath: /etc/ssl/certs
  • TLSv1.2 (OUT), TLS handshake, Client hello (1):
  • TLSv1.2 (IN), TLS handshake, Server hello (2):
  • TLSv1.2 (IN), TLS handshake, Certificate (11):
  • TLSv1.2 (IN), TLS handshake, Server key exchange (12):
  • TLSv1.2 (IN), TLS handshake, Server finished (14):
  • TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
  • TLSv1.2 (OUT), TLS change cipher, Client hello (1):
  • TLSv1.2 (OUT), TLS handshake, Finished (20):
  • TLSv1.2 (IN), TLS handshake, Finished (20):
  • SSL connection using TLSv1.2 / ECDHE-ECDSA-CHACHA20-POLY1305
  • ALPN, server accepted to use h2
  • Server certificate:
  • subject: C=US; ST=CA; L=San Francisco; O=Cloudflare, Inc.; CN=skylon.com
  • start date: Oct 15 00:00:00 2018 GMT
  • expire date: Oct 15 12:00:00 2019 GMT
  • subjectAltName: host “test.skylon.com” matched cert’s “*.skylon.com”
  • issuer: C=US; ST=CA; L=San Francisco; O=Cloudflare, Inc.; CN=Cloudflare Inc ECC CA-2
  • SSL certificate verify ok.
  • Using HTTP2, server supports multi-use
  • Connection state changed (HTTP/2 confirmed)
  • Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0
  • Using Stream ID: 1 (easy handle 0x5559d76b1810)

GET / HTTP/2
Host: test.skylon.com
User-Agent: curl/7.58.0
Accept: /

  • Connection state changed (MAX_CONCURRENT_STREAMS updated)!
  • http2 error: Invalid HTTP header field was received: frame type: 1, stream: 1, name: [upgrade], value: [http/1.]
  • HTTP/2 stream 1 was not closed cleanly: PROTOCOL_ERROR (err 1)
  • Connection #0 to host test.skylon.com left intact
    curl: (92) HTTP/2 stream 1 was not closed cleanly: PROTOCOL_ERROR (err 1)

Website never outputs.

With http2 disabled:

laura@york:/etc/apache2$ curl -v --http2 https://test.skylon.com

  • Rebuilt URL to: https://test.skylon.com/
    % Total % Received % Xferd Average Speed Time Time Time Current
    Dload Upload Total Spent Left Speed
    0 0 0 0 0 0 0 0 --:–:-- --:–:-- --:–:-- 0* Trying 104.26.4.63…
  • TCP_NODELAY set
  • Connected to test.skylon.com (104.26.4.63) port 443 (#0)
  • ALPN, offering h2
  • ALPN, offering http/1.1
  • successfully set certificate verify locations:
  • CAfile: /etc/ssl/certs/ca-certificates.crt
    CApath: /etc/ssl/certs
    } [5 bytes data]
  • TLSv1.2 (OUT), TLS handshake, Client hello (1):
    } [213 bytes data]
  • TLSv1.2 (IN), TLS handshake, Server hello (2):
    { [106 bytes data]
  • TLSv1.2 (IN), TLS handshake, Certificate (11):
    { [2169 bytes data]
  • TLSv1.2 (IN), TLS handshake, Server key exchange (12):
    { [114 bytes data]
  • TLSv1.2 (IN), TLS handshake, Server finished (14):
    { [4 bytes data]
  • TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
    } [37 bytes data]
  • TLSv1.2 (OUT), TLS change cipher, Client hello (1):
    } [1 bytes data]
  • TLSv1.2 (OUT), TLS handshake, Finished (20):
    } [16 bytes data]
  • TLSv1.2 (IN), TLS handshake, Finished (20):
    { [16 bytes data]
  • SSL connection using TLSv1.2 / ECDHE-ECDSA-CHACHA20-POLY1305
  • ALPN, server accepted to use http/1.1
  • Server certificate:
  • subject: C=US; ST=CA; L=San Francisco; O=Cloudflare, Inc.; CN=skylon.com
  • start date: Oct 15 00:00:00 2018 GMT
  • expire date: Oct 15 12:00:00 2019 GMT
  • subjectAltName: host “test.skylon.com” matched cert’s “*.skylon.com”
  • issuer: C=US; ST=CA; L=San Francisco; O=Cloudflare, Inc.; CN=Cloudflare Inc ECC CA-2
  • SSL certificate verify ok.
    } [5 bytes data]

GET / HTTP/1.1
Host: test.skylon.com
User-Agent: curl/7.58.0
Accept: /

0 0 0 0 0 0 0 0 --:–:-- 0:00:01 --:–:-- 0{ [5 bytes data]
< HTTP/1.1 200 OK
< Date: Thu, 18 Oct 2018 01:56:20 GMT
< Content-Type: text/html; charset=UTF-8
< Transfer-Encoding: chunked
< Connection: keep-alive
< Set-Cookie: __cfduid=d829126e731937d1d656de95c466eb0411539827779; expires=Fri, 18-Oct-19 01:56:19 GMT; path=/; domain=.skylon.com; HttpOnly
< Link: https://www.skylon.com/wp-json/; rel=“https://api.w.org/”, https://www.skylon.com/; rel=shortlink
< Strict-Transport-Security: max-age=63072000; includeSubdomains; preload
< Upgrade: http/1.
< Cache-Control: max-age=600
< Expires: Thu, 18 Oct 2018 02:06:19 GMT
< Vary: Accept-Encoding
< Expect-CT: max-age=604800, report-uri=“https://report-uri.Cloudflare.com/cdn-cgi/beacon/expect-ct
< Server: Cloudflare
< CF-RAY: 46b76345aa7955f4-ORD
<
{ [639 bytes data]
100 90670 0 90670 0 0 71113 0 --:–:-- 0:00:01 --:–:-- 71057

And I get an output of the html code as well (which I’ve omitted.)

Does this shed any light on the issue?

Thanks to all who have been helping me :slight_smile:

problem as i suspected is still the rogue HTTP header = Upgrade HTTP/1 being sent from your origin and being passed on by Cloudflare it seems as you can’t upgrade an already HTTP/2 connection to HTTP/2

http2 error: Invalid HTTP header field was received: frame type: 1, stream: 1, name: [upgrade], value: [http/1.]
HTTP/2 stream 1 was not closed cleanly: PROTOCOL_ERROR (err 1)
Connection #0 to host test.skylon.com left intact
curl: (92) HTTP/2 stream 1 was not closed cleanly: PROTOCOL_ERROR (err 1)

check all Apache origin httpd.conf and apache vhost and .htaccess files for any additionally added Upgrade headers that maybe added

1 Like

Unfortunately, there’s no instance of the word “upgrade” in my /etc/apache, and I have no other config files.

Also, a curl -v -http2 http://www.skylon.com never shows the upgrade header, which makes me think it can’t be passed from the origin server? Wouldn’t it show on the origin server as well? (Which is running http2.)

try a recursive case insensitive grep search in /etc/apache

grep -ir 'upgrade' /etc/apache

can also do search on your apache public web root directories to see if .htaccess files added such

I did do a recursive search… HOWEVER. I fixed it.

My protocols line was

Protocols h2 http/1.

I am 100% unsure why it was like that, but the trailing 1 was missing and that killed it. I re-enabled http2, and ran a cURL again:

  • SSL connection using TLSv1.2 / ECDHE-ECDSA-CHACHA20-POLY1305
  • ALPN, server accepted to use h2
  • Server certificate:
  • subject: C=US; ST=CA; L=San Francisco; O=Cloudflare, Inc.; CN=skylon.com
  • start date: Oct 15 00:00:00 2018 GMT
  • expire date: Oct 15 12:00:00 2019 GMT
  • subjectAltName: host “test.skylon.com” matched cert’s “*.skylon.com”
  • issuer: C=US; ST=CA; L=San Francisco; O=Cloudflare, Inc.; CN=Cloudflare Inc ECC CA-2
  • SSL certificate verify ok.
  • Using HTTP2, server supports multi-use
  • Connection state changed (HTTP/2 confirmed)
  • Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0
    } [5 bytes data]
  • Using Stream ID: 1 (easy handle 0x561d833a0810)
    } [5 bytes data]

GET / HTTP/2
Host: test.skylon.com
User-Agent: curl/7.58.0
Accept: /

{ [5 bytes data]

  • Connection state changed (MAX_CONCURRENT_STREAMS updated)!
    } [5 bytes data]
    0 0 0 0 0 0 0 0 --:–:-- 0:00:01 --:–:-- 0< HTTP/2 200
    < date: Thu, 18 Oct 2018 02:14:14 GMT
    < content-type: text/html; charset=UTF-8
    < set-cookie: __cfduid=df7efc7e273272d29e3efb9d89ed110211539828852; expires=Fri, 18-Oct-19 02:14:12 GMT; path=/; domain=.skylon.com; HttpOnly
    < link: https://www.skylon.com/wp-json/; rel=“https://api.w.org/”, https://www.skylon.com/; rel=shortlink
    < strict-transport-security: max-age=63072000; includeSubdomains; preload
    < cache-control: max-age=600
    < expires: Thu, 18 Oct 2018 02:24:12 GMT
    < vary: Accept-Encoding
    < expect-ct: max-age=604800, report-uri=“https://report-uri.Cloudflare.com/cdn-cgi/beacon/expect-ct
    < server: Cloudflare
    < cf-ray: 46b77d793f4d5594-ORD
    <
    { [942 bytes data]
    100 90670 0 90670 0 0 43094 0 --:–:-- 0:00:02 --:–:-- 43114
  • Connection #0 to host test.skylon.com left intact

I went back through and re-enabled stuff in CF I had disabled (like TLS 1.3), and I will get my co-workers to test tomorrow.

Thank you SO MUCH for the help!!!

1 Like

ah typo ! heh

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.