What are the limits when using workers for file uploads

Developers Workers
1

There are many questions about file uploads and workers in this forum.
After reading many of them I hope I will be able to make it work.
But before I even attempt, I’d like to ask whether this is a good approach.
Workers have their limitations in terms of memory, CPU time and more.
If I allow users to upload files using multipart, will I hit these limits? If I do, is there a way to scale-up or will I have to develop a whole new solution?
What file size can I expect such script to consistently succeed?
The file will be eventually stored on gcp or s3.

2

Not sure if you know this webpage allready but
the Worker Limits Information can be found here.

From this Information i conculde that your Script Size and your transfered data that you need store as Variable in the Memory before you can send it to gcp or s3 should not exceed 128 MB in total.

So it all depend how big your Worker will be in the end.
The Question is does it need be really procesed as a Worker ?

Looks like Cloudflare offers as a additional solution also stream piping to process bigger uploads than the 128 MB memory Limitation.

Why not processing Uploads direct on the Origin Server itself if possible ?
That way you dont have any limitation when it comes to memory and cpu processing limits.

1 Like
3

If you stream the upload, there’s essentially no limit to how large files you can upload, the worker isn’t processing the data, just passing it along.

1 Like
4

Thanks for the reply.
Yes, I know this page.
My main concern was regarding cpu time and overall run time. Upload can take time and I’m not sure if the script will not abort due to timeout or too much cpu time, before the upload can finish.
I don’t use the origin because I don’t have an origin. The workers are my only backend. I don’t see a point of having additional backend when I can practically do anything with it.
It would feel ashame if this use case will make me create an additional backend server.

I might change the product flow and have the client upload directly to s3 (or wasabi) instead of proxy via my workers.

Thanks!