Fastest way to fetch all KV keys in namespace

Hey!

I’m currently fetching all keys in a namespace using the generator below.

Is there a faster way to “bulk fetch” all keys in a namespace? Can this be parallelized somehow?

foobar: async function* () {
  let cursor = true

  while (cursor) {
    const list = await FOOBAR.list(
      cursor === true ? {} : { cursor: cursor }
    )

    cursor = list.list_complete ? false : list.cursor

    for (const key of list.keys) {
      yield await FOOBAR.get(key.name, 'json')
    }
  }
}
1 Like

If you do not need all the data stored, or your data is small, you could try to save it (or a portion of it) as metadata. In this case, LIST operation will get you everything in one request. There is no bulk GET, unfortunately…

Interesting. What are the limits of metadata?

I’m storing about 250-300 characters of JSON as the value currently:

{
   "token":"000000000000000000000000000000000000000000000",
   "plugin":"1.6.0",
   "dropin":"1.6.0",
   "foobar":"4.0.9",
   "framework":"5.4.1",
   "network":"0",
   "php":"7.3.18-1+ubuntu18.04.1+deb.sury.org+1",
   "extention":"5.2.1",
   "binary":"3.1.2",
   "openssl":"7.3.18-1+ubuntu18.04.1+deb.sury.org+1"
}

AFAIK, the metadata limits are not documented (maybe somebody could correct me?). For the data in your example, I think you can store it whole as metadata. You should test, though.

It’s 1024 bytes per key: https://github.com/cloudflare/workers-docs/pull/917

1 Like