Ruby TUS: not able to upload to the video library(>200mb)

[user8131]

I am working with ROR to implement TUS upload(>200mb) I got up to where I get a response with the library URL and able to upload to my server endpoint, however, I am very confused after this.

I am using the endpoint: localhost:3000/files because I just found out today that you cannot use the media library URL that you get from the headers[“location”] as the endpoint… Now the file gets uploaded to the /file path but does not upload the file to the Cloudflare video library. What am I missing here? I cannot find any more documentation on it, ruby-tus only has s3. I would love some help!

Thank you

[renan]

Hey! It seems like you might have implemented a upload server with TUS instead of using it to upload to Stream.

I Google’d around and couldn’t find a TUS client implementation in Ruby. However, I think it can be implemented without a Ruby Gem depending on your comfort with making HTTP requests in ruby.

Roughly the steps would be:

• Make a POST request to Stream at https://api.cloudflare.com/client/v4/accounts/:account_identifier/stream with the required headers: Upload-Length, Tus-Resumable, Content-type. Get the location header from this request.
• Make PATCH requests to Stream at the location you just got from the step above. This request should contain the first 10*1024*1024 bytes of the file you’re uploading (you can use any multiple of 256 bytes, but 10MB is a good start) and Content-Length, Upload-Offset and Tus-Resumable headers.
• Repeat the step above until the whole file is uploaded.

If the first step is successful, you will see a video appear with “pending upload” next to it in the dashboard. Way more detail on the steps above is here: Resumable upload protocol 1.0.x | tus

If you have a web server running in your application, you can try uploading to Stream by having Stream make a request to your web server for the (we call this copy via link). If you have a web page where a users uploads the videos, you can use direct creator uploads with tus-js or Uppy - latter is what we use for the uploads in the dashboard.

Hope this helps you and others to see this. I’m happy to clarify if you have any questions

[user8131]

Thank you so much for your reply, my apologies for not being super clear on what I was doing. I am currently using direct creator uploads.

1. I do a POST request to https://api.cloudflare.com/client/v4/accounts/${accId}/stream?direct_user=true
   and that creates a pending upload in the dashboard
   This is where the confusion happens…

• When you said I am using TUS instead of upload to Stream? what would be the difference?
• with URL that I get from the response from POST request where do I make the PATCH request?

Thanks in advance

projectInput.addEventListener("input", function() {
    const fileTypeVideo = this.files[0].type.includes('video')
    const token        = ""
    const uploadLength =  900000000
    const authEmail    = "email"
    const authKey      = ""
    const accId        = ""

    


    if (fileTypeVideo) {
        // request for one-time tokenzied URL
        async function handleRequest() {
            const init = {
                method: 'POST',
                headers: {
                'Authorization': `bearer ${token}`,
                'Tus-Resumable': '1.0.0',
                'Upload-Length': uploadLength,
                'X-Auth-Email': authEmail, 
                'X-Auth-Key': authKey,
                'Content-Type': "application/json"
                },
            }
            const response = await fetch(`https://api.cloudflare.com/client/v4/accounts/${accId}/stream?direct_user=true`, init)
            const results = await gatherResponse(response)


            return new Response(null, {headers: {'Access-Control-Expose-Headers':'Location','Access-Control-Allow-Headers':'*','Access-Control-Allow-Origin':'*','location': results}})
            }
            

            async function gatherResponse(response) {
                const { headers } = response
                return await headers.get('location')
            }

            handleRequest()
            .then((res) => {
                const endPoint = "http://localhost:3000/files/"
                console.log(res.headers.get("location"))

                const options = {
                    endpoint: endPoint,
                    headers: {
                        'Access-Control-Expose-Headers':'Location',
                        'Access-Control-Allow-Headers':'*',
                        'Access-Control-Allow-Origin':'*',
                        'location': res.headers.get("location")
                    },
                    chunkSize: 50 * 1024 * 1024,
                    resume: true,
                    onError: function(error) {
                        console.log("Failed because: " + error)
                    },
                    onProgress: function(bytesUploaded, bytesTotal) {
                        var percentage = (bytesUploaded / bytesTotal * 100).toFixed(2)
                        console.log(bytesUploaded, bytesTotal, percentage + "%")
                    },
                    onSuccess: function() {
                        console.log("Download %s from %s", upload.file.name, upload.url)
                    }
                }

                const upload = new tus.Upload(document.querySelector(".custom-file-upload").files[0], options)
                
                upload.start()

                // start uploading here
            })
            

            

        console.log("video")
    } else {
        console.log("not a video")
    }
    
});

[renan]

Your code seems to be creating a direct creator upload in the browser and then uploading the video to your RoR server.
The general flow should go like this:

1. You make a request to Stream to create a direct creator upload in your RoR app.
2. You pass the URL returned to the browser.
3. The browser starts the upload using tus and this URL instead of http://localhost:3000/files/. The URL allows the upload to go directly to Stream.

[user8131]

Hi Renan,

I am guessing that this is what you mean:

handleRequest()
        .then((res) => {
            const endPoint = res.headers.get("location") < this is the location URL from cloudflare

            const options = {
                endpoint: endPoint,
                headers: {
                    'Access-Control-Expose-Headers':'Location',
                    'Access-Control-Allow-Headers':'*',
                    'Access-Control-Allow-Origin':'*',
                    'location': res.headers.get("location")
                },
                chunkSize: 50 * 1024 * 1024,
                resume: true,
                onError: function(error) {
                    console.log("Failed because: " + error)
                },
                onProgress: function(bytesUploaded, bytesTotal) {
                    var percentage = (bytesUploaded / bytesTotal * 100).toFixed(2)
                    console.log(bytesUploaded, bytesTotal, percentage + "%")
                },
                onSuccess: function() {
                    console.log("Download %s from %s", upload.file.name, upload.url)
                }
            }

            const upload = new tus.Upload(document.querySelector(".custom-file-upload").files[0], options)
            
            upload.start()

When I do this I get the following error:

Access to XMLHttpRequest at ‘https://upload.videodelivery.net/tus/e25252d2255756a9d3636260744a5568?tusv2=true’ from origin ‘http://localhost:3000’ has been blocked by CORS policy: Request header field access-control-allow-origin is not allowed by Access-Control-Allow-Headers in preflight response.

[renan]

You don’t have to set the access-control-expose-headers, access-control-allow-headers, access-control-allow-origin or location headers in the request to upload! TUS should set the right headers for you.

Here’s a codepen example from @zaid that might help https://codepen.io/cfzf/pen/wvGMRXe

[user8131]

So if I put the URL as a endpoint the upload should be successful right? I’ve tried multiple time and does not work…

[renan]

I’m sorry about that. Do you see any specific errors you can share? Did you take a look at the codepen above?

[user8131]

[zaid]

In the codepen demo, the URL you enter should point to your backend endpoint which returns the upload url in the location header. The default url when you open the codepen demo calls the following worker script (which essentially uploads a video to my account):

    addEventListener('fetch', event => {
event.respondWith(handleRequest(event.request))
})

/**
* Respond to the request
* @param {Request} request
*/
async function handleRequest(request) {
const init = {
    method: 'POST',
    headers: {
    'X-Auth-Email': '---',
    'X-Auth-Key':'---',
    'Tus-Resumable': '1.0.0',
    'Upload-Length': request.headers.get('Upload-Length'),
    'Upload-Metadata': request.headers.get('Upload-Metadata')
    },
}
const response = await fetch("https://api.cloudflare.com/client/v4/accounts/YOUR_ACCOUNT_ID/media?direct_user=true", init)
const results = await gatherResponse(response)
return new Response(null, {headers: {'Access-Control-Expose-Headers':'Location','Access-Control-Allow-Headers':'*','Access-Control-Allow-Origin':'*','location':results}})

}

async function gatherResponse(response) {
const { headers } = response
return await headers.get('location')

}

[user8131]

Did you look at the previous conversation that I had with Renan?

[user8131]

I’ve been struggling with this for days now… I need some solid solution to this issue that I am having.

Thanks

[user8131]

This is the next thing I’ve tried

So not the endpoint is pointed to my rails backend, “/video_upload” routes once it hits the routes

def file
    respond_to do |format|
        msg = { :status => "ok", :message => "Success!",  
               'Access-Control-Expose-Headers' => 'Location',
               'Access-Control-Allow-Headers' =>'*',
               'Access-Control-Allow-Origin' => '*' }
        format.json { render :json => msg, :location => params[:location] }
    end
end

When I do this I get the following error:
new:1 Access to XMLHttpRequest at ‘https://upload.videodelivery.net/tus/3d750d15f961dccdbc569d47d69ebd69?tusv2=true’ from origin ‘http://localhost:3000’ has been blocked by CORS policy: Request header field x-csrf-token is not allowed by Access-Control-Allow-Headers in preflight response.

any suggestions?

[user8131]

This is the next thing I’ve tried

So not the endpoint is pointed to my rails backend, “/video_upload” routes once it hits the routes

def file
    respond_to do |format|
        msg = { :status => "ok", :message => "Success!",  
               'Access-Control-Expose-Headers' => 'Location',
               'Access-Control-Allow-Headers' =>'*',
               'Access-Control-Allow-Origin' => '*' }
        format.json { render :json => msg, :location => params[:location] }
    end
end

When I do this I get the following error:
new:1 Access to XMLHttpRequest at ‘https://upload.videodelivery.net/tus/3d750d15f961dccdbc569d47d69ebd69?tusv2=true’ from origin ‘http://localhost:3000’ has been blocked by CORS policy: Request header field x-csrf-token is not allowed by Access-Control-Allow-Headers in preflight response.

any suggestions?