I have been facing quite a weird issue. I have a DNS record sets on Cloudflare with Proxied Status ON which allows the end user to download the files which are larger than 1GB in size.
The files fail to download if end-users have download speed lower than 8 MB/sec which is 64 Mbps. End users with 64 Mbps of download can easily download the complete file.
However, any user with lower than this speed couldn’t download. The download fails exactly at 1GB if any user have download speed below to 8MB/sec.
Please find below the scenario for your reference in which I have tested it:
1- The desktop used for testing has 1 Gbps of download speed.
2- I used IDM to restrict the download bandwidth for observing the behavior.
3- When keeping the download limit set to 8MB/sec or above, the file downloads properly.
4- Downloading the same file on 7.5MB/sec or below terminates the session exactly at 1 GB and through network error.
5- The browser is kept same for all downloads which is Google Chrome latest version.
6- I am sure its not a connection issue because the file is downloaded as soon as I exceeded the download speed to at least 8MB/sec.
It doesn’t work when DNS record isn’t proxied so all the testing was perform while keeping the proxied status up on that record set.
Not knowing if meassured “1Gb” as 1GB, 1GiB or 1Gb, this leavs a little room for variances.
Since 1000MB / 8MB/s = 125s and this is around the magical 100s timout. It could be the case, that with 8MB/s download you exactly can download the file within the 100s and with 7.5MB/s you run into a timeout.
I am not aware of any of these timeouts for downloading, but as @KianNH mentioned, this could apply at the origin connection. I will set up a 5GB file (to be sure) and try to download it @7.5MB/s.
Will repot back.
What HTTP protocol (“1.1” or “2”) version are you using for the origin connection?
any settings that define timeouts on your webserver?
Well, sometimes, it’s the server which doesn’t support higher speeds, so the user can’t download faster despite better connection speed from his ISP.
Nevertheless, from my experience, if you have to download such a large files via Web Browser, make sure you are serving them to the consumers via unproxied (DNS-only) hostname while using Cloudflare for your domain name.
otherwise, use chunks or make sure to set arguments like read-timeout and tries while using SSH wget or similar on Linux …
else, use FTP, rsync, scp …
Other way would be to upload/download from a hostname like some CDN storage BackBlaze, etc.
I am not so much technical, but I think there is some timeout on HTTP/2 at least over TCP.
Also, happens the same when I try to download a large file - 15GB from Google Drive (NOTE: Google is not using Cloudflare) via Web Browser. It just says not complete, then I hit the button “refresh”, then it continues downloading it until it’s finished.
not to mention anti-virus software being “man-in-the-middle” of the connection, etc.
The file size is 1.4GB and the speed on which the download runs successful is 8MB/sec. Ideally, with this speed the file download won’t take more than 125sec which you have mentioned but what I’ve noticed in real-time, it is not taking more than 46 - 50 seconds for overall download to complete.
1- Its HTTP/2 version for origin connection.
2- I will ask developers for this.
3- Can you please elaborate it further as how can I test it?
(DNS-only) hostname while using Cloudflare for your domain name.