It’s my environment too. But I didn’t have any updates when the problem occurred. I updated only 2 days later to eliminate possible problems.
OK - one thing to check is to cURL directly to your origin
curl https://example.com --insecure --verbose --connect-to ::1.2.3.4 --header "accept-encoding: br"
Replace example.com with your hostname and 1.2.3.4 with your origin IP. Then, look in the response for the response header content-encoding: br and then look to see if the response body is encoded or returns human-readable HTML. If it is returning human readable HTML - this is likely the source of the problem - your origin is “claiming” to be returning a brotli encoded website, but is not actually encoding.
1 Like
I would generate an HAR file, but that would mean my site would go down again. And after two days of having this issue I’m not sure that’s a risk I’m willing to take at this time. I’ve uploaded a screenshot of what I was seeing when everything was unresponsive, so you can see what things look like. It seemed like all javascript was being blocked and most of the CSS. But as you can see from the screenshot, it looked like it was trying to load some sort of “choose your language” form field.
1 Like
i tried on windows command prompt.
and yes i can see the html directly as output.
But it gives the output directly so i couldnt check response header content-encoding: br.
Brotli is disabled when i tried.
If you scroll up the output, the headers should appear at the top of the cURL, because of the --verbose flag.
What we think may be happening is your origin is telling us the response is brotli encoded but it isn’t actually, causing us to fail. We’re testing some new functionality in some locations and I think this is what is causing the problem. The workaround right now would be to stop whatever it is on your origin that’s incorrectly saying your responses are brotli-encoded when they are not.
2 Likes
@chris.botkins let me copy my reply from the other potentially related thread:
OK - one thing to check is to cURL directly to your origin
curl https://example.com --insecure --verbose --connect-to ::1.2.3.4 --header "accept-encoding: br"
Replace example.com with your hostname and 1.2.3.4 with your origin IP. Then, look in the response for the response header content-encoding: br and then look to see if the response body is encoded or returns human-readable HTML. If it is returning human readable HTML - this is likely the source of the problem - your origin is “claiming” to be returning a brotli encoded website, but is not actually encoding.
2 Likes
i see, so disabling brotli should solve the problem in this case.
C:\Users\conta>curl https://domain.com.br --insecure --verbose --connect-to ::15.235.117.000 --header "accept-encoding: br"
* Connecting to hostname: 15.235.117.000
* Trying 15.235.117.000:443...
* Connected to (nil) (15.235.117.000) port 443 (#0)
* ALPN: offers h2,http/1.1
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):
* TLSv1.3 (IN), TLS handshake, Certificate (11):
* TLSv1.3 (IN), TLS handshake, CERT verify (15):
* TLSv1.3 (IN), TLS handshake, Finished (20):
* TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.3 (OUT), TLS handshake, Finished (20):
* SSL connection using TLSv1.3 / TLS_AES_256_GCM_SHA384
* ALPN: server accepted h2
* Server certificate:
* subject: CN=domain.com.br
* start date: Feb 8 17:17:51 2023 GMT
* expire date: Feb 7 17:17:51 2024 GMT
* issuer: O=AO Kaspersky Lab; CN=Kaspersky Anti-Virus Personal Root Certificate
* SSL certificate verify result: self-signed certificate in certificate chain (19), continuing anyway.
* using HTTP/2
* h2h3 [:method: GET]
* h2h3 [:path: /]
* h2h3 [:scheme: https]
* h2h3 [:authority: domain.com.br]
* h2h3 [user-agent: curl/7.88.1]
* h2h3 [accept: */*]
* h2h3 [accept-encoding: br]
* Using Stream ID: 1 (easy handle 0x1b01efb79c0)
> GET / HTTP/2
> Host: domain.com.br
> user-agent: curl/7.88.1
> accept: */*
> accept-encoding: br
>
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* old SSL session ID is stale, removing
< HTTP/2 200
< expires: Thu, 19 Nov 1981 08:52:00 GMT
< cache-control: no-store, no-cache, must-revalidate
< pragma: no-cache
< content-type: text/html; charset=UTF-8
< link: <https://domain.com.br/wp-json/>; rel="https://api.w.org/"
< link: <https://domain.com.br/wp-json/wp/v2/pages/272847>; rel="alternate"; type="application/json"
< link: <https://domain.com.br/>; rel=shortlink
< etag: "2404-1691601512;br"
< x-litespeed-cache: hit
< content-encoding: br
< vary: Accept-Encoding
< content-length: 46146
< date: Wed, 09 Aug 2023 17:19:41 GMT
< server: LiteSpeed
<
Warning: Binary output can mess up your terminal. Use "--output -" to tell
Warning: curl to output it to your terminal anyway, or consider "--output
Warning: <FILE>" to save to a file.
* Failure writing output to destination
* Connection #0 to host (nil) left intact```When I remove the header from the command, it returns readable HTML.
Are you still able to replicate this issue right now?
Our team has made some rollbacks and would like to know if you are still experiencing this.
1 Like
The issue occurs randomly at various times during the day. We will continue monitoring the situation tomorrow.
In response to Simon’s suggested workaround, we’ve taken the step of disabling brotli from our nginx servers. However, we’ll need to wait for approximately 1-2 days to determine if this action resolves the problem.
In our case, it might be harder to figure out what exactly solved the problem because we’ve turned off brotli on our own servers. So, it’s not easy to say if the solution is only due to your rollbacks. We’re watching closely, and if users who haven’t turned off brotli let us know the issue is gone, that could really help us understand what’s happening.
Thanks for your ongoing help as we work this out.
1 Like
Are you still able to replicate this issue right now?
Our team has made some rollbacks and would like to know if you are still experiencing this.
I’m reposting the response I wrote in the other thread here, for those who aren’t following
https://community.cloudflare.com/t/cut-off-half-pages-randomly/542880/23?u=tn12
2 Likes
Curious that in my case, according to curl’s answer, there is no problem related to brotli. So we have the same final problem, but for different “reasons”.
I created a ticket at litespeed support and they informed me that they also received complaints about the same issue.
I questioned them, because another colleague reported in the topic that his http server was litespeed (same as mine). But you just informed that yours is ngix.
I haven’t made any changes and it’s been a few hours since I’ve received any complaints from my customers. But I am attentive to any complaints so that I can attach the HAR.
2 Likes
It is better to test by lots of people with high usage tomorrow because it is hard to replicate it. We test it with a big team. Sometimes it occurs one after another, sometimes with long durations. Thanks for your effort.
Do we need to make changes to lightspeed server too for brotli disabling? We will test without any change on server but i disabled brotli on cloudflare panel only.
Hey people,
I am finding more reports of people facing the same issue, and each of them are testing and finding results of various degrees.
For some reason, this has not happened on any zones on the Business Tier.
I have seen it on Free (mostly) and (Pro).
If you are seeing issues, could you open a ticket for us to track this please.
(please provide a ticket number in your response)
Thank you.
1 Like
Ticket ID: 2889769
1 Like
The new code on our side was rolled back by 15:45 UTC yesterday. So you should no longer need to disable brotli at your origin, but it’s still a good idea to fix your origin’s broken behaviour. Either by ensuring content is actually brotli encoded, or by disabling brotli entirely if you can’t get it working at your origin server.
Generally speaking if your origin is telling an eyeball or a proxy that something is encoded and then not encoding it, it will cause unpredictable behaviour.
If you can still replicate this issue after 15:45 on the 9th August, then please do provide the information we previously asked for - HAR file, cURL etc.
3 Likes
I’ve merged the two related topics to keep things in one place since I believe all reports are related to the same problem with bad origin brotli behaviour.
Our team will reply shortly but my response above applies to your origin server also.
Today we didnt get that problem, pages load normally. Brotli is disabled on my cloudflare panel. I will open it later to check.