Bulk Writing to KV?

#1

Is it possible to add keys in bulk over the HTTP APIs to the Workers KV store?

I see I can have a billion records up there, but serially, via curl, it takes about a second to upload. I’m not looking to push a billion, but I am looking to improve performance. I can parallelize my calls, but it would be nice if I could initialize the KV store. Possible?

#2

You can write directly from a worker:

With this, you can write function that import for example a CSV directly to KV’s.

Just keep in mind the limit though, one write per second per unique KV key.

#3

That seems like an interesting approach. That would work. Thanks. A little unfortunate in my case because I then need to secure an endpoint and potentially make calls in reverse which I will also need to authenticate.

#4

It was just an example, even with CSV I’d recommend converting to JSON and run a loop on the content and update a key on each loop. If you have the data somewhere else, you can just login, download it using the worker and the do the update inside the worker.

1 Like