Workers Site / webpack

All,

I’ve deployed my first site with Workers Sites, although I’ve done a lot with Workers the past six months or so. I’m not a JS dev, so I seem to be missing a lot of simple / assumed stuff.

My quick question is, I’m trying to add my existing logging infrastructure (JS from other Workers already deployed) to my Workers Site. My other projects use a Makefile to concatenate the individual JS files into a single script for uploading. Here, I’m trying to do it the “Wrangler way” :slight_smile: . How on earth do I incorporate multiple JS files into the final worker/script.js?

package.json's “files” doesn’t seem to do it, and I’m having trouble verifying (due to webpack) that import in index.js is doing what it says. I’ve not created separate production/development stacks yet, so I’m not able to just throw it up on CF and see what happens yet.

To that end, How do I tell webpack not to minify the output? I tried placing webpack_config = ... in wrangler.toml, and putting it in development mode, but wrangler borked on that saying that Sites doesn’t support webpack config… ?

And more importantly, if webpack minification messes up a stack trace, how do I disable minification without a webpack config?

There’s a fix for the “Sites doesn’t support webpack config”, but it’s not released yet.

See here:

You can install it by installing rust and compile wrangler from source.

EDIT: Also, Workers don’t support line-breaks in source code, that’s why minification is required. It’s also about size, even small NPM packages are big when un-packed source and comments are included.

Sweet! Thanks! I’m not sure how my google-fu failed me, but it might’ve been the Super Bowl “festivities” :slight_smile:

Really? Which type of line-breaks? I’ve been pushing amalgamated JS, not minified, up to Workers for 6 to 8 months. It’s had comments, line breaks, and everything, but no Node modules. my stack traces even include correct line numbers. (but now they only occur when unit testing exception handling :wink: ) This has been on macOS editing with Vim, so it might be CR-LF v. CR difference?

@jason28 Not sure, I’ve yet to find any setting on my Linux Dev server that works with Workers and Webpack when it’s un-minified and use NPM modules. If you have any tips, I’d be very glad - it would greatly simplify debugging in CF editor.

Well, it seems as long as you’re using Wrangler / webpack / node.js modules, you can’t do it easily. I just tried.

However (warning: not much help for existing projects), if you just concatenate all your JS files manually (I use Make), and then use the CF API to push the single script, it works fine. My final scripts aren’t that big, and I don’t even bother scraping the comments out. I can also confirm that it attaches the KVs to the script nicely as well. The scriptname is what governs which routes it’s attached to, so I set that up through the website interface.

Basically, CF Workers is happy to take a straight, readable, unminified JS file and use it. You get clear stack traces to boot. :slight_smile: Unfortunately, something inside the ~330 packages that make up wrangler, or wrangler itself, prohibit uploading it that way. :frowning:

FTR: here’s how I upload

#!/bin/bash

# NOTE: a whole bunch of code eliminated for brevity / clarity

metadata="$1"
scriptname="$2"
scriptname="${scriptname/.js/}"
 base_url="https://api.cloudflare.com/client/v4/accounts/$CLOUDFLARE_ACCOUNT_ID"
s_url="$base_url/workers/scripts/$scriptname"

curl ${CURL_DEBUG} -X PUT "$s_url" \
    -H "X-Auth-Email: $CLOUDFLARE_AUTH_EMAIL" \
    -H "X-Auth-Key: $CLOUDFLARE_AUTH_KEY" \
    -F "[email protected]$metadata;type=application/json" \
    -F "[email protected]${scriptname}.js;type=application/javascript"

Where metadata.json is:

{
  "body_part": "script",
  "bindings": [
    {
      "type": "kv_namespace",
      "name": "ATTRS",
      "namespace_id": "XXXXXX"
    },{
      "type": "kv_namespace",
      "name": "LUT",
      "namespace_id": "YYYYY"
    },{
      "type": "kv_namespace",
      "name": "CACHE",
      "namespace_id": "ZZZZZZ"
    }
  ]
}

That binds the KVs to that script, and potentially a WASM blob as well.

1 Like

Thanks a ton for sharing! Maybe we can open an issue on Wrangler to have this as an option?

I think it would be easier to just write a much smaller utility in Rust to accomplish the task and avoids the whole webpack / node.js complexity nightmare.

However, there’s a legit usecase (even though I don’t use it) for node.js modules, so figuring out how to concatenate those without blindly using the JS tools to do it would be kind of tricky. I’ve managed to avoid needing to do it so far, but the couple of times I researched it, came up blank. But sometimes me and $searchengines don’t get along. They don’t understand regex. :-/

2 Likes

Agreed, I’d also love a simpler packager since i only use ES6 modules anyway.

I’ve looked at Parcel to replace webpack, but constantly got broken packages for some reason I didn’t have time to dive into… One tends to avoid messing with bundling when it often consume hours or days of tinkering.

I would imagine a Rust based bundler being much much faster too, but it might not be.

My only reason for saying Rust was because I’ve just finished one project in it, and it would be neat to do a second. However, I don’t think I’m up for the time sink nor ■■■-pain that JS internals would involve.

Anyone willing to do the work can write it in whichever language they prefer, as long as it does what’s needed.

Honestly, I’d rather see Workers evolve into just running Rust in WASM and skip the whole JS piece. You can bloat up a project quickly in Rust also (see wrangler), but it’s a lot more disciplined overall than JS ever could be. I’d love to know if WASM can directly interact with the KV store, and if it can do the equivalent of fetch(). If so, then I might just migrate to that.

Ha! I got the black dots. meh.

Yes, it can. https://github.com/jRiest/the-best-goats/blob/master/src/lib.rs#L48-L60 is an example of the KV binding.

Holy ■■■■! That’s awesome! I’m not sure how useful managing pictures of goats is, but that’s exactly what I was hoping for, thanks for the pointer!

3 Likes

After a little bit deeper dive into the code, I have a question for the CF folks:

index.js: appears to be doing a lot of heavy lifting in JS to make a “rust-only worker” a reality. e.g. wrapping and passing fetch(), allocating memory for the heap allocator, among other non-traditional JS tasks.

What’s the possibility of those knobs being moved to native code in the V8 engine so as to avoid the JS?

index.js is created by wasm-bindgen; it needs to do these things currently, because

This is going to happen someday, but is not in the wasm spec yet.

Ah, that explains it. Bad assumption on my part. I make it a habit not to commit generated objects/files. So I assumed that if it was in the tree, it was written by hand.

Regardless, I look forward to seeing the WASM spec extended in this regard.