What if Workers could access your SQL/NoSQL database?

Hello everyone! I’m on the Workers team and we’re trying to gather some feedback on the future of storage in Workers. We’re considering letting Workers access an external database… would you be interested in this? Here are some questions to get you started… we want to hear all of your ideas!

  • What would you want to build with it?
  • What databases should we support?
  • Do you want us to cache queries, how would you want to tune it?
  • What could we do to make databases… just easier?
5 Likes

Honestly, I think the eventually-consistent KV is suitable for many usecases. However, a similar product, but immediately consistent, which was just for small info, integers and such, might be useful. I’m thinking along the lines of treating each instance of a Worker like a thread, and having some sort of mutex / cmpexchg mechanism. If the resource is a) small, and b) not disk-backed, it might be doable.

Some things that this theoretical db could enable are:

  • reference counting
  • mutual exclusion / locking
  • global stats counters
  • global state machine latching

As for what one could do with this capability? Well, I don’t like spouting buzzwords, but a blockchain’s state could be incremented and tracked globally without a centralized server.

There’s a lot of other fun decentralized systems problems that could be solved and deployed if that tool existed…

I rambled about one problem which it would solve:

https://community.cloudflare.com/t/global-decentralized-leader-selection-with-workers-kv-was-fetch-with-a-client-certificate/82883

Thanks,

Jason.

3 Likes

Personally, most NoSQL databases I’d use are exposed over HTTPS, to which workers are already well suited. I’d rather see iteration on Workers KV to get closer to something like DynamoDB, with LevelDB-style prefix queries or mutation-based worker triggers.

2 Likes

+1, already can expose whatever we want from our database through api, and will have to find a way to handle scaling, a-z availablity etc so it hard for me to imagine what problem it will solve that we dont already have(direct access to db)

if if you manage the db for us it another story

The idea is interesting but I the implementation would likely be a pain and nowhere near as performant as Cloudflare Workers are. So I’m not interested.

I’d rather see improvements to Workers KV so that those can behave more like a database.

1 Like

+1 for better KV. More features like Redis and fast writes!

1 Like

I agree with the general sentiment, improve KV’s instead.

@jason28 listed some really needed cases like global stats counters, locking etc.

I’d also like to see it possible to do time-series logging directly inside workers writing directly to KV.

Because logging in general is expensive and hard, doing it in a worker makes it even harder and unpredictable (having to rely on timeouts). Having it in workers makes it easy, fast and predictable.

4 posts were split to a new topic: Logflare Discussion