KV - Implementation details

Hi,

I’ve got a couple of questions about how KV is implemented under-the-hood which I wasn’t able to find answers for online.

  1. How are keys ordered when using list?

If I execute a request such as NAMESPACE.list({"prefix": "user:1:"}), and there are many keys with this prefix, which order are they returned in?

Crucially, is the order deterministic? Are they sorted lexicographically, by order of insertion or using some other method?

  1. How does the eventual consistency model work?

I understand that when modifying an existing key that there will be a short period of time directly afterwards where reads may return different versions of the object depending on which replica serves the request.

Is this also true of the first-time write to a specific key? Some systems offer strong consistency for the initial write of a key, but eventual consistency for subsequent writes. I’m assuming this is not the case for KV?

Bonus third question :wink: From your experience what are some worst-case timings for a value to achieve consistency across your whole cluster?

3 Likes

I’m interested in the CF response to this, and i’m not certain about this, but Workers KV [might] use Quicksilver as described in this blog post, which would mean:

On average we hit a p99 of 2.29s for a change to be distributed to every machine worldwide.

But again the blog post doesn’t specifically mention Workers KV itself using Quicksilver to distribute changes, so hopefully CF staff will give some insight :slight_smile:

1 Like

Hi @missinglink,

Thanks for the questions!

  1. Keys are returned in lexicographically sorted order according to their UTF-8 bytes. The ordering is deterministic, they will always be returned in the same order.

  2. Writes will always be visible globally within one minute. They will often be visible sooner, depending on factors such as whether the key has been accessed before and how frequently it’s being accessed. But you can always count on updates being visible everywhere within a minute.

  3. One minute :slightly_smiling_face:

2 Likes

To add some context, KV does not use quicksilver:

Workers KV is distinct from Quicksilver; I have no idea if it’s going to be open source or not, sorry!

https://news.ycombinator.com/item?id=21095022