We here at Cloudflare are experimenting with making SQL databases available for easy communication from the edge. It will allow you to query and insert data into your databases from within a worker. I wanted to gauge interest in this from the community & hear any interesting use cases you would want to try experimenting with (we’ll reach out to you via email if you’d like to try it out).
Where would these databases be housed?
That would enable a ton of use-cases, we could store snippets of HTML, JSON for configurations and make relational queries. The question is, would it cost per request to it? Or would you charge for amount of data stored? In any case, I’m interested in trying it out!
The idea would be to install Cloudflare services next to your DB server, we wouldn’t be housing your databases for the time being. We are actively exploring data storage within Cloudflare as well, however, the expected timing for that to be released is not as immediate as this.
Great to hear you have some interesting use cases. The cost associated with the requests are associated with the current cost of using Argo Tunnel (https://support.cloudflare.com/hc/en-us/articles/115000224192-Billing-for-Argo). We wouldn’t be storing the data, routes to your existing DB server. Hopefully this helps, and I’ll be sure to reach out soon, when we’re ready to for users to try it out.
I haven’t grasped the concept of Workers. And when I tried out Argo on a client’s site, it made that site slower. So this might not be for me. But I’ll keep an open mind and look for interseting use cases and ideas.
Oh, I thought it was going to be as easy and reliable as workers - basically a function in the worker.
What makes this unique rather than just connecting to any distributed database already out there?
In the near future Cloudflare Workers will have the ability to write into Cloudflare’s Cache (globally distributed) so that you can retrieve data faster than from your DB server. This will be a benefit of using Workers compared to distributed databases already out there.
The other component here is that SQL databases (MySQL, PostgreSQL) don’t have a simple HTTP interface to communicate over. We are enabling customers to have this easy communication with Workers, as many valuable applications of Workers require persistent data storage, which we currently don’t provide the best availability for.
You are right that many NoSQL databases & other distributed databases already have this capability (Workers can leverage these already ). In the long-term if/when we decide to offer a distributed database of our own, we would plan for native support with Workers.
Worker and streaming JSON
I’m sorry to hear that, this is actually quite a rare case so I would recommend contacting Cloudflare Support for assistance with using Argo for site speed benefits.
Luckily, this solution is somewhat parallel to your current issues as it is to help with rapid prototyping of solutions for clients (less hassle to build your applications entirely in a Worker).
Thanks for keeping an open mind & I look forward to hearing about any interesting use cases.
That’s exactly what i wanted to hear!
The best of both worlds and then some - do you know when CF Cache writing is coming to workers?
Can’t wait to try that out!
Yup, definitely hoping to provide the best of both worlds & making previously challenging configurations as easy as possible. The date is still TBD for releasing CF Cache writing with Workers, but I’ll be sure to follow up with you as soon as it’s official.
Thanks a lot for the update Arhenn, this will certainly be an exiting year!
I might be a little late to the party, but thought I would throw my 2 cents in.
My first worker that I am writing now, is doing just that. I’m pinging my Neo4j database to pull back user information, validate the request, then create a JWT token.
The long term goal is to have a Single Sign On solution in place.
This seems to be a really interesting use case & I would love to hear more about what you are implementing & if you need any guidance. There are definitely methods to communicate with existing databases through HTTP compatible APIs (For NoSQL/NewSQL databases), but the priority of implementing this system is to communicate with standard SQL databases (not offering standard HTTP APIs). This is a prevalent use-case we’ve seen & we hope our solution will be helpful to other customers very soon. I appreciate your thoughts & will relay them to our group working on Single Sign-On solutions, as this is part of our roadmap.
If you wanted to do a call, we could schedule some time to talk about my project.
I would definitely be interested in this… Currently I use AWS Lambda behind my Worker, but it would be great to remove that step.