Http h2c upgrade requests result in "Connection reset by peer"

Cloudflare is improperly handling h2c upgrade requests over http.

Here’s how to see the issue using curl (this issue is reproducible for any site hosted by Cloudflare):

$ curl http://cloudflare.com --http2 -v
*   Trying 2606:4700::6811:b055:80...
* Connected to cloudflare.com (2606:4700::6811:b055) port 80 (#0)
> GET / HTTP/1.1
> Host: cloudflare.com
> User-Agent: curl/7.73.0
> Accept: */*
> Connection: Upgrade, HTTP2-Settings
> Upgrade: h2c
> HTTP2-Settings: AAMAAABkAAQCAAAAAAIAAAAA
> 
* Recv failure: Connection reset by peer
* Closing connection 0
curl: (56) Recv failure: Connection reset by peer

According to RFC 7540 section 3.2 https://tools.ietf.org/html/rfc7540#section-3.2 a server that doesn’t support the upgrade should simply ignore the headers requesting it.

I suspect Cloudflare implemented this behavior recently as a defense against smuggling: https://lab.wallarm.com/cloudflare-fixed-an-http-2-smuggling-vulnerability/

However, that defense was implemented a bit too aggressively. I agree that h2c upgrade requests that occur over https should result in this behavior, but h2c upgrade requests that occur over http should not.

Can Cloudflare please fix this issue?

BTW, this issue was originally discovered by Kodi: https://github.com/xbmc/xbmc/pull/16601 And also reported to curl: https://github.com/curl/curl/issues/6099

Thanks!

1 Like

Let’s go for the full monty

@MVP @cscharff @adspedia @amayorga @dpolaske @Peter-CF @ryan

Probably need to contact Cloudflare tech support via the ticket system

curl-http3 http://www.cloudflare.com --http2 -v                        
*   Trying 2606:4700::6811:d209:80...
* Connected to www.cloudflare.com (2606:4700::6811:d209) port 80 (#0)
> GET / HTTP/1.1
> Host: www.cloudflare.com
> User-Agent: curl/7.73.1-DEV
> Accept: */*
> Connection: Upgrade, HTTP2-Settings
> Upgrade: h2c
> HTTP2-Settings: AAMAAABkAAQCAAAAAAIAAAAA
> 
* Recv failure: Connection reset by peer
* Closing connection 0
curl: (56) Recv failure: Connection reset by peer
curl-http3 http://www.cloudflare.com --http2 --http2-prior-knowledge -v 
*   Trying 2606:4700::6811:d109:80...
* Connected to www.cloudflare.com (2606:4700::6811:d109) port 80 (#0)
* Using HTTP2, server supports multi-use
* Connection state changed (HTTP/2 confirmed)
* Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0
* Using Stream ID: 1 (easy handle 0x564dce471620)
> GET / HTTP/2
> Host: www.cloudflare.com
> user-agent: curl/7.73.1-DEV
> accept: */*
> 
* http2 error: Remote peer returned unexpected data while we expected SETTINGS frame.  Perhaps, peer does not support HTTP/2 properly.
* Empty reply from server
* Closing connection 0
curl: (52) Empty reply from server

though it does seem Cloudflare HTTP/2 only supports APLN h2 and http/1.1 and not h2c so probably why. But I see what you mean h2c not supported should be ignored on non-https connections. @LucasCF might have an idea?

curl-http3 -I https://www.cloudflare.com --http2 -v                          
*   Trying 2606:4700::6811:d109:443...
* Connected to www.cloudflare.com (2606:4700::6811:d109) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
*  CAfile: /etc/ssl/certs/ca-certificates.crt
*  CApath: none
* TLSv1.2 (OUT), TLS handshake, Client hello (1):
* TLSv1.2 (IN), TLS handshake, Server hello (2):
* TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):
* TLSv1.3 (IN), TLS handshake, Certificate (11):
* TLSv1.3 (IN), TLS handshake, CERT verify (15):
* TLSv1.3 (IN), TLS handshake, Finished (20):
* TLSv1.3 (OUT), TLS handshake, Finished (20):
* SSL connection using TLSv1.3 / TLS_AES_128_GCM_SHA256
* ALPN, server accepted to use h2
* Server certificate:
*  subject: businessCategory=Private Organization; 1.3.6.1.4.1.311.60.2.1.3=US; 1.3.6.1.4.1.311.60.2.1.2=Delaware; serialNumber=4710875; C=US; ST=California; L=San Francisco; O=Cloudflare, Inc.; CN=cloudflare.com
*  start date: Oct 30 00:00:00 2018 GMT
*  expire date: Nov  3 12:00:00 2020 GMT
*  subjectAltName: host "www.cloudflare.com" matched cert's "www.cloudflare.com"
*  issuer: C=US; O=DigiCert Inc; OU=www.digicert.com; CN=DigiCert ECC Extended Validation Server CA
*  SSL certificate verify ok.
* Using HTTP2, server supports multi-use
* Connection state changed (HTTP/2 confirmed)
* Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0
* Using Stream ID: 1 (easy handle 0x564f74062620)
> HEAD / HTTP/2
> Host: www.cloudflare.com
> user-agent: curl/7.73.1-DEV
> accept: */*
> 
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* old SSL session ID is stale, removing
* Connection state changed (MAX_CONCURRENT_STREAMS == 256)!
< HTTP/2 200 
HTTP/2 200 

The client should fail before attempting the connection to the server.
HTTP2 requires https, curl should throw a message HTTP not supported for http2 http3 connections.
Browsers do this “gracefully”.

That’s not entirely accurate. Browsers decided to enforce SSL but HTTP 2 per se does not require it, I think (but might be wrong) that changed with HTTP 3.

@sandro is right.
The requirement was on the table then cancelled, but then major Browsers force HTTPS for HTTP2 requests. Maybe Cloudflare operates the same way if it is HTTP2 has to be over TLS.

1 Like

IMHO they should either serve it on HTTP then or issue a redirect to HTTPS or at least send a proper response, just failing though is something they should fix.

IMHO :male_detective:
I would prefer the server to respond with a 412 or general 400.
An HTTP redirect will always leak metadata on the connection.

Sure, but it will do so already on the request and if the client chose HTTP then HTTP should be served.

IMHO they should just serve it under HTTP. If not possible, issue a redirect to HTTPS. If that is not desired either at least return a 505.

1 Like

That would also violate the RFC. Please see RFC 7540 section 3.2 https://tools.ietf.org/html/rfc7540#section-3.2
If Cloudflare doesn’t want to do h2c, that’s perfectly fine; they should follow the RFC and ignore the upgrade request, proceeding as normal with HTTP/1.1 handling.

¡Hola y’all!

I’ve reviewed on this with the Protocols team here at Cloudflare.

They mentioned that a fix is already in the works and should be available in due course:

we’ll discard those h2c upgrade headers instead of force close the connection.

Please let us know if you have any further questions here and we’ll be happy to help.

¡Gracias!
Alex

4 Likes

Cheers @amayorga for the update :+1:t2:

Thank you!
Was scratching my head about this

This topic was automatically closed 5 days after the last reply. New replies are no longer allowed.