For urls (eg https://google.com), I get this error thrown: TypeError: Provided readable stream must have a known length (request/response body or readable half of FixedLengthStream)
Is that because the fetch response does not contain a Content Length header? What should I do in this case, catch the exception and try something like env.MY_BUCKET.put('somekey, await response.text())
?
Better ideas?
Thanks!
Assuming you’re trying to fetch and store HTML, I’d fallback to text() like you said. However, you could check if response.headers.get('content-length') returns anything as opposed to catching an exception.
You’ll want to include the response headers you got in your put operation so that the content-type is correctly brought along, otherwise it’ll end up like this.
Pass them through to the options optional parameter of put as the httpMetadata object - like so…
let body;
if (response.headers.get('content-length') == null) {
body = await response.text()
} else {
body = response.body
}
await env.BUCKET.put('test', body, {
httpMetadata: response.headers
});
Notably, httpMetadata is comprised of various content-* and cache-* headers.