Upload files to S3 via Cloudflare


#1

Hi

I would like to know if I can upload files to a private AWS S3 bucket through Cloudflare’s CDN edge location (aka a reverse CDN). My goal is to improve data transfer speed and improve connection reliability especially in China. My needs are:

  1. The PUT request will have all the necessary header auth keys required by the S3 bucket which Cloudflare needs to forward if the request is to succeed.
  2. Cloudflare does not need to cache the files or if we do, it needs to honor the permission settings of the file in S3.
  3. Needs to have good performance and network reliability in China mainland even though the S3 bucket in located in USA. Must not be affected by the Great Firewall.

Is this possible with Cloudflare and what are the steps I need to do to implement this?

I tried a similar thing using AWS Cloudfront and while the performance did improve, the connection reliability was poor due to DNS resolution issues. So I’m confident this can work if your network is better there.

Thank you,
Hiren.


#2

So a few things.

  1. Access to Cloudflare’s China network isn’t a standard part of any plan offering and is typically only available as an add-on to an Enterprise plan and requires the customer have an ICP license.
  2. Cloudflare can improve the performance and reliability of assets served in China and potentially of items retrieved from outside of China which would be cached on our China network.
  3. Cloudflare’s service in China is not a magic bullet which bypasses or otherwise circumvents the Great Firewall. That is 100% not a service we provide.

So Cloudflare could serve as an SSL endpoint for the client for whatever uploads they were doing. Cloudflare acts as a straight proxy for outbound requests so if the request included the authentication/authorization we would pass it along and we don’t cache or otherwise accelerate uploads. Any request from China outside the network would 100% be subject to the Great Firewall.

If all you were interested in was upload performance, I’m not sure the China network would make much of a difference. Name resolution might be better, but beyond that I’m not sure I see any straightforward improvements it would provide.


#3

I’m only interested in improving upload performance by using it as a proxy.

Could you elaborate as to why you don’t see it would improve performance?

I’m hoping for better name resolution and connection reliability to S3 because sometimes S3 is blocked or it seems like it’s being rate limited. AWS CloudFront showed a 2-3x upload time improvement but connection to edge location was very spotty. Since you have edge locations in China, connection between the client to edge location should be good. Then the question is how much is the connection between your edge location and S3 US is affected by the Great Firewall?


#4

The connection for a client to an edge location should be good you’re correct. So in that sense it can potentially be an improvement in performance / reliability. We’re still going to make requests from our edge out of China in a manner which as far as I know is still subject to passing through the Great Firewall. Obviously we’ll have larger pipes and be able to reuse connections potentially more efficiently than individual clients would so that could also be a performance benefit.

We definitely see improved reliability for our customers who use the Cloudflare network in China, it’s just that most often the focus is on serving content to clients rather than upload/ transfer out so I have fewer data points in that regard as to potential benefits in speed.


#5

Thank you for the explanation. It’s definitely a unique case with very few data points - makes it challenging.

This topic can be closed.

Regards,
Hiren