Calling Web Worker from within a CF Worker?


I’m having great success with Workers and Workers KV. So much so that I’m now moving on to optimizations.

I have a small nginx server to which my CF Worker scripts send log messages to. This has worked great for development and debugging, but it’s not ideal. Each call to my log function create a whole new connection to the log server, and there’s no ordering of messages.

Ideally, I’d like to solve this by standing up a Dedicated Web Worker thread at the start of my script, which opens one connection to my log server. Then, each call to write a log message actually sends a message to the Web Worker. That message is then forwarded to my log server.

It looks like this is possible in JS save one detail. All the examples do setup like:

    var worker = new Worker(`thread.js`)

Where thread.js is a separate script file on the server. Unfortunately, that doesn’t apply here. At least, I don’t think it does…

So my question is:
a) How do I declare a separate script file in my script?
b) How else can I establish a background thread for a single, sustained I/O connection?



Do you mean hold a connection open, like a Comet?

AFAIK there is no way to maintain a connection pool with workers but I could be wrong and would actually love to know how.

If you’re just trying to log stuff and don’t want to setup a server you can use Logflare or some other saas logging service where you don’t have to bother with a server.

Comet looks like WebSockets, which is really just a hack with a bandaid with some duct tape to work around the fact that http was never intended to be bidirectional. It was purely for document fetching. If you want a bi-directional socket, remove http and just use the underlying TCP. Either side can write to the socket with reckless asynchronous abandon. :slight_smile:

Regardless, I’m basically looking to optimize the following scenario: Right now, every time my script generates a log message, it opens up a fresh connection to my webserver and sends the message through it. So, if a single instance of my script runs to completion, that’s anywhere from 3 to 7 log entries, which means 3 to 7 http connections (webserver is behind an Argo tunnel). Which… doesn’t scale as I move towards deployment.

So it seems I have three choices here:

  1. Use LogFlare or similar
  2. Coalesce outgoing messages into a single connection (which is what my OP was aimed at)
  3. Figure out how to fire off a raw UDP message to my syslog server, which is designed to handle this kind of stuff without all the overhead of TCP/http.

I really don’t like (1), as this is data I want to get under my custody and backup. It’d be great if I could just do (3), but I’ve not seen any hint of that so far. Maybe wasm code can open a UDP socket?



Quick update:

It looks as though the Chrome v8 JS engine supports creating TCP and UDP sockets (and even multicast UDP :slight_smile: ) via chrome.sockets.udp

I just tried it in a test script and it barfed with workers.api.error.internal_server… oh well.



That’s why I asked, it’s been taken up before and there’s no spec for it in service workers.