Data center location when using KV in Durable Objects

I was just curious about this behavior. Combining the two statements below, it would appear that Durable Objects can use KV with “nearly” zero latency?

I imagine there would be some race conditions introduced by mixing the two products.

Is there any documentation on this? Has anyone tried it in the wild? @gmckeon?

KV

Changes are immediately visible in the edge location at which they’re made, but may take up to 60 seconds to propagate to all other edge locations.
Source: https://developers.cloudflare.com/workers/learning/how-kv-works

DOs

Requests for a Durable Object id are routed by the Workers runtime to the Cloudflare point-of-presence that owns the Durable Object.
Source: https://developers.cloudflare.com/workers/learning/using-durable-objects

Durable Objects can use KV with the same latency characteristics as any other Worker using KV.

If a given Durable Object is the only thing reading and writing a given KV value, yes you could read your writes immediately afterwards in the same location, but at that point why not use Durable Object storage?

It’s more useful for a Durable Object to be the only thing writing a given KV value, and for other Workers in other locations to read that value from KV. That will still involve delay, but it doesn’t increase the likelihood of KV race conditions, and will support higher read rates than reading directly from a single Durable Object.

2 Likes

The reason would be that KV can store 25mb and DO can only store 32kb.

Basically I’m imagining DOs making updates to KV values.

If the DO is the only thing allowed to write to the KV, then it should be able to read the values back immediately.

However, the KV doesn’t have the input/output gates that DO has, so I do think you would run into some subtle race conditions in production.

So it could happen that:

  • DO req_1 - KV get
  • DO req_2 - KV get
  • DO req_1 - KV put
  • DO req_2 - KV put // overwrite doesn’t include updates from DO req_1

One way to avoid this is having only 1 client talk to 1 DO at a time, or use a cron trigger to do the updates to the KV.

A more complicated approach would be to use the DO storage to gate the KV put, but that’s a bit involved.

Does this sound crazy? How else would you bring the atomic power of DO to KV, if you want to write larger amounts of data?

Sure, large values makes sense.

Maybe I’m misunderstanding your example, but req_1 and req_2 are in the same DO right? So they have access to the same memory and won’t be running in different threads, so you can implement whatever semantics you want regarding what the value in memory should be before it’s written to KV.

You have been a tremendous help :slight_smile:

I think my key concern came down to how the KV was being accessed. This is kind of important if you have a multi-user app where everyone is editing a document. You can’t trivially read from KV for each user on each request, because they’ll end up with older, eventually consistent data.

You’re last comment is particularly helpful for understanding the model I’m going for. I would have everyone editing the in-memory DO data structure and then writting to KV. Due to the nature of DO input/output gates this would basically just mean the last write wins, which is fine for my use case.

I suppose if the DO is evicted, I can also use:

state.blockConcurrencyWhile(async () => { ... load data from KV ... })

In the DO constructor to load the data from KV back into memory and block any requests until this is done.

Thanks for your help!