Workers KV Durability/Replication - Safe to use as independent DB?

I haven’t found anything about this in the documentation.

I really like the Workers KV concept. But I am a bit unsure how safe the data is, in terms of durability. For example an infrequent read key/value is only stored in one datacenter. What happens if the disk that stores the kv crashes? Are the values internally replicated to multiple disks?

I was thinking to use KV workers to actually store some user data and not just cache them. This would require a reasonable replication, even for infrequently read data.

Is there any information about this?

Calling the usual persone here… @sklabnik!

I presume there is some sort of duplication, just don’t know how much.

1 Like

From what I know there is replication at the Core of KV, presumably across datacenters. Although the workers script moves to the edge, KV only moves to some PoPs due to demand, read request frequency.

For safeguard, if you are worried of Cloudflare Encryption you could use the Web Crypto API on your Worker to Encrypt and Decrypt KV.

“KV achieves this performance by being eventually-consistent. New key-value pairs are immediately available everywhere, but value changes may take up to ten seconds to propagate. Workers KV isn’t ideal for situations where you need support for atomic operations or where values must be read and written in a single transaction.” – Cloudflare Docs

“Workers KV read performance is determined by the amount of read-volume a given key receives. Maximum performance for a key is not reached unless that key is being read at least a couple times per minute in any given data center.” – Cloudflare Docs

1 Like

We’ve designed KV to be similar to other cloud storage products in this regard. For example, S3, Firebase Cloud Storage, etc.

4 Likes

Thanks for the feedback @sklabnik. As far as I know, these products usually replicate the data at least across 3 separate disks, depending on the product they will store the data at least in 3 different regions. Is this something you can confirm for Workers KV as well?

Thanks for the hint. I haven’t thought about that yet.

I’m guessing that since they mention S3, the data durability is in the 11x9s.

https://aws.amazon.com/s3/faqs/#Durability_.26_Data_Protection (Scroll up a bit)

In other words, there’s an extremely little chance of data-loss.

Would gladly have this confirmed though… @sklabnik

1 Like

I can confirm that we do things for durability but cannot talk about the specifics, because we reserve the right to change exactly how we implement it and don’t want to guarantee we do things in a particular way.

2 Likes

Makes sense, thanks @sklabnik.

Is this a use case that workers KV is designed for or is it mostly designed for “less critical data”? The examples in the documentation seem to talk about less critical data (e.g. cached from the main application).

It’s intended for any durable data; the docs tend to talk about less critical things because that’s been a big use-case, but it’s not an ephemeral store.

3 Likes

Thanks for the further explanation @sklabnik.

2 Likes