Google Cloud Storage Bucket as Origin


#1

I can’t find any documentation on how to set up a Google Cloud Storage bucket as the origin server. Both Google (https://cloud.google.com/interconnect/docs/how-to/cdn-interconnect) and Cloudflare (https://www.cloudflare.com/integrations/google-cloud/) mention this program, but there’s no info anywhere on how to set this up. Please help.


#2

Hey there. Do you want to serve static content from a bucket like HMTL/images? Have you seen this article?
https://cloud.google.com/storage/docs/hosting-static-website

I’m not certain that the CDN interconnect is supported for buckets.


#3

I’d like to serve static content like images, JavaScript & CSS files, and maybe PDF files. I thought the whole point of the CDN Interconnect program was to support Google Cloud Storage buckets. That’s exactly the way it works with Fastly, for example: https://docs.fastly.com/guides/integrations/google-cloud-storage


#4

As it relates to Cloudflare specifically? I’ve done some research and information is quite lacking, so I understand your frustration. That Fastly link is exactly the kind of documentation Cloudflare should have regarding this service, if they do indeed support it for Buckets, as their page only mentions “Google Cloud Platform” - not any specific service like Compute Engine, App Engine, etc.

This is of particular interest to me because I am currently migrating images and other static resources from S3 to GCP buckets for hosting and I would like to take advantage of this service if it’s available.

I’m hoping someone from Cloudflare can provide a bit of clarification here.


#5

Thank you, @andy. The Cloudflare terms regarding this kind of configuration are also confusing. Here’s what it says:

SECTION 10: LIMITATION ON NON-HTML CACHING
You acknowledge that Cloudflare’s Service is offered as a platform to cache and serve web pages and websites and is not offered for other purposes, such as remote storage. Accordingly, you understand and agree to use the Service solely for the purpose of hosting and serving web pages as viewed through a web browser or other application and the Hypertext Markup Language (HTML) protocol or other equivalent technology. … Additionally, the purpose of Cloudflare’s Service is to proxy web content, not store data. Using an account primarily as an online storage space, including the storage or caching of a disproportionate percentage of pictures, movies, audio files, or other non-HTML content, is prohibited.

Does this mean that I can’t use Cloudflare to just link to JS, CSS, and image files from within my web pages?

Someone at Cloudflare please clarify.


#6

Hey @widerwebs. It’s good you’re reading the ToS. I think what really needs to be clarified here is the phrase:

disproportionate percentage

Defining this is likely extremely difficult as I’m sure there are a number of factors in play, so I don’t really expect anyone from Cloudflare to provide any sort of figures. A general idea of what type of site that may violate this might be helpful though.

E.g. Photo gallery website serving ~2,000 images totaling ~350MB. Personally I do not consider this example to be disproportionate percentage when considering total size in relation to datacenter capacity (info that I don’t believe would be publicized). Though I could be way off base here in my consideration.

I think in general you are ok with your JS and CSS as these are typically very small files.

Edit: Additionally, I think it would be insightful to know if the criteria for determining ‘disproportionate percentage’ is different based on plan level.


#7

Anyone from @cloudflare able to help us out?


#8

Hello @widerwebs,

No worries here, this section of the ToS is protecting us against evident abuse of our service with websites that consume a lot of bandwidth, a lot of cache space and not at all being traditional websites (Using HTML language with styles, images, JS…).
You’re not supposed to provide a streaming service, for example, you’re not supposed to provide a Dropbox-like service as well. The data storage notion is slightly different than the pure storage of vital content for your website (referred in your HTML code).

It doesn’t mean that you’re only limited to use the HTML language with us :wink: (and hopefully), it’s just mean that you engage yourself to not abuse our service.
No public numbers to give but what we consider an abuse in our service is really massive in term of bandwidth or GB stored in our cache instances, so I assume that you’re safe to onboard your website in CloudFlare if you’re looking at our self-serve plans.


#9

Thanks for providing some examples of abuse and relative scope of size. This info is definitely helpful! :nerd_face:


#10

Hello @stephane thanks for reaching out. I’m actually asking about not just hosting the JS, CSS, and images for a single website but for many websites that I host to use. I’ll have perhaps 50,000 websites linking to these files (all used in web pages, not just for storage). Is this legit?


#12

@stephane there would be one domain total (no subdomains). There wouldn’t be a zone created for each site. Would that be fine?


#13

@widerwebs, hum. Using us as what we’re then, like a CDN + Security :slight_smile:

So I understand that you have a bunch of websites that you don’t plan to onboard on Cloudflare but you’ve for all of those some common contents that you want to transit through us via a third party domain playing the role of the CDN for your child domains.

This is not abusing our ToS a per that explanation, what is the estimated bandwidth per month?

NOTE: on the Cloudflare domain configuration, you’ll need to deactivate the hotlinking protection because we’ll block all of the requests with a referer which are not corresponding to the secured and accelerated domain.


#14

I guess if you’re planning to build/ offer a storage service for others where they could upload their images for use on a webpage… uh we’d need to talk. If you’re talking about building something like an ad network? Again, let’s talk, we can probably help you do some really interesting things with our service. Want to create your own high volume cdn-esque environment like https://cdnjs.com/? um… probably ok maybe? But we should probably talk depending on the data volumes to make sure we have a plan that makes sense for all parties.

Honestly you could talk to @stephane or myself (amazingly our email is our forum username @cloudflare.com) in more detail and we’ll be happy to answer as honestly as we can without a hard sales pitch (though we are both sales/solutions engineers so sometimes we can’t help ourselves). :smiley:


#15

Thank you, @stephane and @cscharff for your responses.

You’re both exactly right, I’d like to use Cloudflare as a CDN (with a Google Cloud Storage bucket as the origin). I’m assuming you wouldn’t allow me to pay just $20/month for a Pro account if I have more than 30 GB of bandwidth monthly. Right? Or is that okay for the $20 plan?


#16

Not trying to sound flippant… but at 30GB a month we probably wouldn’t even notice unless you’re making hundreds of millions of requests for a 1k file… in which case I will hunt you down and beat you with a feather. :wink:

If you’re getting into the TBs per month range we start to get into a gray area where we should make sure everyone is getting the fair value and service.


#17

so…many…jokes…:thinking: