Using the "Streaming Responses" for secure file delivery


#1

Hi CF Team!

I am really enjoying exploring the Workers functionalities and I am starting to imagine all the possibilities that they will enable us to do.

So here is my idea to which I would like to get your opinion if it is a good case for Workers, and if it’s reasonable to expect it to work within the CPU time boundaries.
I have a project with some files that have to be served to users. Files are being stored on several file servers. Eg.

  • fileserver01.domain.xyz/file-01.mp3
  • fileserver02.domain.xyz/other-file-07.jpg

But, for security reasons, we don’t want the end user to know the exact server/URL path where the files are being stored (like the ones above).
That’s why we are using a reverse-proxy server to handle the requests, lookup the proper file server for particular file, and serve it back to the user. All that is being handled through a common domain, the links that users see and use are eg.

  • files.domain.xyz/file-01.mp3
  • files.domain.xyz/other-file-07.jpg

Do you think it’s possible to use Worker to replace the reverse-proxy server, request the proper file server name using some cached JSON data file and start passing through the file data from the storage server to the browser, hiding the real file location from the visitor?

As files can be several tens of megabytes big, and the download could take some minutes or an hour, I guess the Streaming Responses approach to pass-through the file data stream would be the best way to go?

Thanks for your opinions and any advice in advance!


#2

Hi! This is not just possible, but probably easier than you imagine!

You can absolutely load a JSON file in your Worker which contains the file mapping. That file will be cached by our CDN on the same edge nodes your Worker is running on, so it should load very quickly.

As long as you don’t need to modify the responses at all, you don’t even need to explicitly do anything to make them stream. Responses stream by default!

Finally, while we require you to begin any subrequests within the first 15 seconds of your Worker’s execution, there is no limit on how long the actual response can take to stream to the client, so there’s no problem there either.


#3

Hi Zack!

Thanks for the really quick reply.

Great, I am very happy that my idea is indeed possible, and an easy one to implement. Now I just have to read the docs completely and (hopefully) write the proper Worker code :slight_smile:

No, I don’t need to change the response body, I only have to hide from the visitor that the source was a backend server. It should to look like the file comes from “virtual subdomain” ( eg. files.domain.xyz in my example) …

Thanks,
Bernard