Cloudflare Stream Beta Feedback

@hostmaster4 good question. Those limits do not apply to Stream video uploads. During the beta, the video size limit is 2 GB.

Yesterday and today, still getting this errors. Uploads mostly don’t work. Every once and a while a video successfully gets on to stream. Mostly I get timeouts like the one below after upload, during encoding I imagine:

Error 504 Ray ID: 3e5dd94a2e1d994f • 2018-01-31 15:59:28 UTC
Gateway time-out
You
Browser
Working
Los Angeles
Cloudflare
Working
api.cloudflare.com
Host
Error
What happened?
The web server reported a gateway time-out error.
What can I do?
Please try again in a few minutes.
Cloudflare Ray ID: 3e5dd94a2e1d994f • Your IP: 66.214.155.10 • Performance & security by Cloudflare

Any ideas?

1 Like

Like I presumed, my Access Key was wrong. Even though I’ve checked it multiple times, Cloudflare always gave me a key that was identical to my Zone ID. Today I checked again and the key had changed. With the new Access Key I could at least get the upload started. So there’s seems to be a bug with the CF admin area showing the wrong Access Key.

Both upload attempts with tus-upload and tus-upload-js fail however, exactly as described already by @hostmaster4.

tus-upload

[2018-02-02 10:15:08,132] INFO Uploading 1049253 bytes chunk from offset: 4194304
Traceback (most recent call last):
File “/usr/local/bin/tus-upload”, line 9, in
load_entry_point(‘tus.py==1.3.1’, ‘console_scripts’, ‘tus-upload’)()
File “/Library/Python/2.7/site-packages/tus.py”, line 99, in _cmd_upload
offset=0)
File “/Library/Python/2.7/site-packages/tus.py”, line 203, in resume
offset = _upload_chunk(data, offset, file_endpoint, headers=headers)
File “/Library/Python/2.7/site-packages/tus.py”, line 239, in _upload_chunk
response=response)
tus.TusError: Upload chunk failed: Status=504

tus-upload-js gets stuck around the last uploaded chunk.

It should be fixed by Wednesday. Sorry for the delay!

@alexosoft That upload 504 error should be fixed by wednesday and I’ll post here once it’s fixed.

Hey folks! We just deployed an update that speeds up uploads and fixes most issues above. Just testing from my laptop, I saw about a one fifth the time for the same video file.

If you are using the tus.py client, we ask you to add the flag --chunk-size=5242880 when uploading and if you’re using an other tus compliant uploader, you will need to set the minimum chunk size to 5MB (5242880 bytes). You will get a 501 error is the chunk size is too low. This restriction does not apply if your file is smaller than 5MB.

I’ve updated the google doc to show the change in the upload command.

1 Like

We signed up a LONG time ago and can’t wait to “be powered by Cloudflare Stream”. We just received an email from Andy. Soon! Very soon! :smiley:

2 Likes

Awesome!

Oliver Yu | Engineering Manager
[email protected]
https://www.cloudflare.com/

1 888 99 FLARE | www.cloudflare.com

The API looks pretty much as expected and as advertised earlier. I miss the Rest API documentation I got a link to earlier.

  1. We have a community with user generated content. We have to give end-users the upload enpoint without the upload url revealing any api keys or can be misused if exposed?

  2. We need to put some limits on user upload so end-users do not try to upload hours of video. Can we set a max length and/or size so the video is either rejected during upload or in processing?

  3. Is there a setting to keep origin file? Or are only encoded files stored?

  4. Are there callbacks/webbhooks so we can get status updates without pulling the api constantly?

  5. How are thumbnails created and selected? Our current service provides 10 thumbnails sampled from key frames and we can radiolyssnarna pick one of them as preview thumbnail through the api or upload a custom image.

  6. Possibly related to this, we would like a feature where you get samples from the video (by iterating over thumbnails) when you hover with your mouse over the preview (in the future).

  7. Will you support authentication of videos? So a video-file/player can not be hotlinked or shared.

  8. Will there be support for custom branding of the player? Any visible Cloudflare or Bitmovin branding in the player would be a showstopper.

  9. Analytics? How do we hook player events into our own view counting api and Google Analytics?

  10. And finally, the most important question, what’s the price? I need some sort of indication before we can commit resources…

Thanks!

Hi @qxtra, glad you are trying out the Stream beta!

  1. We have a community with user generated content. We have to give end-users the upload enpoint without the upload url revealing any api keys or can be misused if exposed?

We have a neat solution to this — Cloudflare has a product called Workers that allows you to write JavaScript that runs at the Cloudflare edge. Using Workers, you can embed your API credentials on the fly so that your API key is never distributed to your clients.

I’ll add some sample code you can copy paste later on.

  1. We need to put some limits on user upload so end-users do not try to upload hours of video. Can we set a max length and/or size so the video is either rejected during upload or in processing?

Configurable max length/size of video - great idea. We don’t have it yet but maybe we should add it at some point. For now you can do the validation on the client side. Before sending us the video you can check the size of the file against your requirements.

  1. Is there a setting to keep origin file? Or are only encoded files stored?

Stream stores both the original file and the encoded versions by default.

  1. Are there callbacks/webbhooks so we can get status updates without pulling the api constantly?

Webhooks to get status updates. We agree - this will be so much better than polling - at some point we will probably add this.

  1. How are thumbnails created and selected? Our current service provides 10 thumbnails sampled from key frames and we can pick one of them as preview thumbnail through the api or upload a custom image.

Currently the thumbnail selected is the first frame of the uploaded video. Over time we will allow you to say which frame from the video should be used.

  1. Possibly related to this, we would like a feature where you get samples from the video (by iterating over thumbnails) when you hover with your mouse over the preview (in the future).

Yes! We will add.

  1. Will you support authentication of videos? So a video-file/player can not be hotlinked or shared.

Yes. What we will do is allow you to specify on which domains should be allowed to serve a given video. This list of domains will be maintainable over API so that you can automate adding and removing domains if you require that.

  1. Will there be support for custom branding of the player? Any visible Cloudflare or Bitmovin branding in the player would be a showstopper.

The player will not be Cloudflare branded at launch. Over time we will allow you to add your own customizations like your logo.

  1. Analytics? How do we hook player events into our own view counting api and Google Analytics?

We provide an analytics API right now (https://developers.cloudflare.com/stream/analytics/) but not a way to collect custom events from the player.

  1. And finally, the most important question, what’s the price? I need some sort of indication before we can commit resources…

Price isn’t finalized and we’ll keep you posted as we know more. @asilverstein may have more to share there.

1 Like

Will the list be processed in order or random? I feel like we signed up during the first few days. I guess maybe we were rejected.

What is the best product to purchase in order to (maybe) convert to Cloudflare Stream when it’s usable?

@qxtra For your question #1: how to use the API without putting API keys on the client, you can use Cloudflare Workers to embed your credentials on the fly.

For an overview of Workers, you can check out https://developers.cloudflare.com/workers/

To get started with Workers, go to the Workers tab of the Cloudflare dashboard: https://www.cloudflare.com/a/workers/

Some Workers sample code you can use for this is:

addEventListener('fetch', event => {
  event.respondWith(proxyMediaRequest(event.request))
})

const myZoneAPI = "www.example.com/client/v4";
const cfAPI = "api.cloudflare.com/client/v4/zones/{zone-id}";

function translateResponseLocation(headers) {
  const newHeaders = new Headers(headers);
  if (headers.has("location")) {
       newHeaders.set("X-Original-Location", headers.get("location"));
       let location = headers.get("location");
       let newLocation = location.replace(cfAPI, myZoneAPI);
       newHeaders.set("location", newLocation);
  }
  newHeaders.set("Access-Control-Allow-Credentials", "true")
  return newHeaders;
}

async function proxyMediaRequest(request) {
  var re = new RegExp("/media/(.*)$");
  const init = {
    method: request.method,
    headers: new Headers(request.headers),
  };
  if (request.method != "GET" && request.method != "HEAD") {
    init.body = request.body;
  }
  init.headers.set("X-Auth-Key","XXXXXXX");
  init.headers.set("X-Auth-Email", "[email protected]");
  var response;
  try {
    var matchArr = re.exec(request.url);
    var subPath = "";
    if (matchArr) {
        subPath = matchArr[1];
    }
    // Change do your zone
    var newURL = "https://api.cloudflare.com/client/v4/zones/{zone-id}/media/" + subPath;
    response = await fetch(newURL, init);

    return new Response(
        response.body,
        {
            status: response.status,
            statusText: response.statusText,
            headers: translateResponseLocation(response.headers)
        }
    )
  }
  catch (e) {
    if (e.response) {
      return e.response
    }
    return new Response("Error", {"status": 500});
  }
}
2 Likes

I have done my first upload test. Upload and encoding worked fine.

I do get an error message when I try to look at the thumbnail or preview link.

Error 526 Invalid SSL certificate
Ray ID: 3ec3d7b3797c7696
Ray ID: 3ec3e52cffb17606

https://qXXXXXr.com/cdn-cgi/trace
fl=128f13
h=qXXXXXr.com
ip=92.244.XX.XX
ts=1518484341.344
visit_scheme=https
uag=Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_3) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/64.0.3282.140 Safari/537.36
colo=ARN
spdy=h2
http=h2
loc=SE

Any suggestion how to proceed?

We have a community where users can upload videos. Current solution store the videos ID in our application database and we can add title, description and other application specific information.

With this Cloudflare we would probably use our own database and store UID, thumbnail and other metadata to be able to quickly produce a list of videos/gallery without hammering on Cloudflare’s API.

I guess this is on your todo list already, but duration is really metadata about the video you should be able to pull out through the api when the video is encoded.

I can predict that we want thumbnail in several sizes so we can use srcset on the img-tag for the thumbnail image.

I think we also would also like to have access to the original filename for the upload in metadata.

Have you considered using some sort of tags? I can see a scenario where we tag uploads with a userid to be able to list all videos with a specific tag quickly or delete all videos from one user by using a tag.

1 Like

@qxtra this is a great question! We also noticed this bug a few days ago and working on fixing it as we speak. As a temporary solution, you can place a page rule at yoursite.com/cdn-cgi/* path to use Full (not strict) or Flexible SSL.

1 Like

@qxtra for your second question: we’re working on adding features to organize your videos like the ones the ones you’ve mentioned.

In the meanwhile, you can use the metadata field in the API as a arbitrary key value where you can store tags and user id information. You can use this either with the optional metadata fields in the tus protocol or right after the upload using the API. See https://tus.io/protocols/resumable-upload.html#upload-metadata for more information on uploading videos with custom metadata

Thanks @dani for the suggested solution with a worker. It might be acceptable for a test or proof-of-concept but we are quite sceptical to Workers as a solution until it matures more.

Most video transcoding libraries I have worked with have an API to create a one time upload endpoint server-side. You can then safely pass this endpoint to a browser (or app) that can upload the video file - without having access to any Cloudflare credentials.

It is a very common scenario. I think that in the long run you must implement a more permanent solution to this scenario to make the service competitive.

2 Likes

100% yes.

Workers – we are using Workers for edge side logic of some of our newest products. I think you can use it with confidence.

Testing the video player embed code locally, I’m getting a CORS error. Is it possible to configure this somewhere and e.g. allow “[domain].local”?

1 Like

Just testing this in the admin panel,

Fast uploads, really fast encodes, amazing playback, actually better using your video, than jumping around in VLC

We are just testing this for content delivery of some of our in house videos / cctv.

Do you have info on access to these urls (can we combine it with access?)
What will be your pricing policy on this, as obviously there is a fair amount of storage about to be used!