I am trying to write 7MB of data using KV bulk insert. Theoretically, the two bits of code below should do the same thing.
The first code just puts in one item using a for loop and the second tries to bulk insert the same item. The code was modified to support debugging, I am obviously trying to insert the entire array of data. When using the entire array, the first fails for load issues after several hundred inserts.
The second one fails with the error shown at the end for just one item in the batch.
for(const item of json) {
try {
const result = await fetch(`https://api.cloudflare.com/client/v4/accounts/$ACCOUNT/storage/kv/namespaces/$KV/values/${item.key}`,
{
method: "PUT",
headers: {
"Content-Type": "application/json",
"X-Auth-Key": "$KEY",
"X-Auth-Email": "$EMAIL"
},
body: JSON.stringify(item.value)
}
);
console.log(result);
} catch(e) {
console.log(e);
break;
}
break; // just break after first since we are debugging
}
fetch("https://api.cloudflare.com/client/v4/accounts/$ACCOUNT/storage/kv/namespaces/$KV/bulk",
{
method: "PUT",
headers: {
"Content-Type": "application/json",
"X-Auth-Key": "$KEY",
"X-Auth-Email": "$EMAIL"
},
body: JSON.stringify([json[0]])
}
)
.then((response) => console.log(response))
.catch((e) => console.log(e))
// Always throws Response {
url: "https://api.cloudflare.com/client/v4/accounts/92dcaefc91ea9f8eb9632c01148179af/storage/kv/namespaces...",
status: 400,
statusText: "Bad Request",
type_: "default",
trailer: Promise { <pending> },
headers: Headers { date: Sat, 23 May 2020 15:01:41 GMT, content-type: application/json; charset=UTF-8, set-cookie: __cfduid=dde913c46802ba06261c26ca0640b11fa1590246101; expires=Mon, 22-Jun-20 15:01:41 GMT; path=/; domain=.api.cloudflare.com; HttpOnly; SameSite=Lax, cf-ray: 597fa754fc3fe35e-SEA, cache-control: no-store, no-cache, must-revalidate, cf-request-id: 02e3a6e9160000e35e4c93e200000001, vary: Accept-Encoding, cf-cache-status: DYNAMIC, expect-ct: max-age=604800, report-uri="https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct", x-envoy-upstream-service-time: 8, server: cloudflare },
body: Body { contentType: "application/json; charset=UTF-8", locked: false, body: [Circular] },
type: "default",
redirected: false
}