Custom TS build - I don't want webpack

Wrangler is shipped with webpack.

I want to use a custom build for my TypeScript project.
(shared code, specific procedures, etc.)

How can I achieve this ?

Is it possible to provide my build endpoint or command somewhere ?

What values should be filled in wrangler.toml
type = “…”
webpack_config = “…”

1 Like

You can change the type to javascript to have it point to your already-built file.
The path of the javascript file can be specified as main in your package.json.

2 Likes

Hello,

Thanks for the fast reply !

So

wrangler.toml
type = “javascript”
webpack_config = “…”

package.json
“main”: “index.js”,

I’m wondering about pointing index.js.

Does it means the result of the build MUST be a single file (bundle) ?

If so, it implies I cannot do dynamic imports, such:

if (x) { await import('sub-file.mjs'); } // import conditionnally

…and I MUST use static imports, such:

import a from 'sub-file.mjs'; // always import, even if not needed

…which is generally considered a bad practice in terms of performance and memory management

I would prefer to still be allowed to use dynamic imports, if possible.

1 Like

It’s not possible for a worker to use script files conditionally. A worker has to consist of a single file, this is why webpack is generally recommended as it allows you to easily pack all your files into a single bundle.
Of course you can still use a different builder other than webpack, as long it produces a single javascript file.

V8 / W3C specification allow modules in workers, shared workers and service workers

const a = new Worker('worker.js', { type: 'module' });
const b = new SharedWorker('/worker.js', {  type: 'module' });
const c = navigator.serviceWorker.register('/sw.js', {  type: 'module'});

I suppose the limitation comes from CloudFlare, that might use a custom implementation of the Worker interface for their Edge Servers.

Maybe I can deploy worker subfiles along with the PWA, so they are available somehow.

Solution 1 - Local import

const m = await import('../site/functions/ae12f6-my-module.mjs');

Solution 2 - Fetch Import

Fetch them, then dynamic import them. Like a dynamic import from the network. Just after fetching them, you should cache them so the next worker can retrieve them from the edge cache.

let modules; // Global
const _import = async (url) => {
  if (typeof modules[url] !== undefined) { return modules[url]; }
  // Fetch url and cache for next worker
  const res = await fetch(url,{cf:{cacheTtl:31536000,cacheEverything:true}});
  // Get a temporary local file url blob:
  let blob = new Blob([await res.text()], {type: 'text/javascript'});
  const blobUrl = URL.createObjectURL(blob);
  // Import and cache for next call
  modules[url] = await import(blobUrl);
  // Free memory
  URL.revokeObjectURL(blobUrl);
  blob = null;
  return modules[url];
}
// At build time, replace all "import(" by "_import("
const m = await _import('https://my-site.app/ae12f6-my-module.mjs');

It is a bit unconventional to publish worker files, meaning server files, on the CDN (must be password free, and I might encrypt those files to ensure secrecy), but that should work.

Thanks for the information !

1 Like

The problem is a security one.

On Workers, you cannot use function generators (which dynamic import requires) because function invocation from strings is disabled. Which also means you can’t use eval either.

If you’re targeting China, you’d want to add your own rate-limiting since Cloudflare charge a lot for that feature and you’re going to be constantly hammered with attacks within China. There’s a few tricks you can do with globals and cloudflare firewall API to handle that.

And I’d also do signed encrypted requests if you do requests to endpoints where DNS pollution can be a problem (very common in china).

1 Like

Good catch.
I understand your point about DNS pollution & encryption.

Do you mean the very itself instruction “await import(’…’)” is disabled in Cloudflare Workers ?

No, dynamic import work but it has to be bundled, it won’t load an external package when needed.

So you can still do things like: const { default: lodash } = await import('lodash');

…Inside of a module or function.

That’s what I do throughout all my Worker apps to add extra isolation. I don’t want all modules to run in a global context (it would be less secure, but also slow down the worker significantly when it’s a big one).

Loading objects into any variable takes CPU-time, which also applies when initiating the script by the worker. That’s why dynamic imports are important in this context. As soon as we reach ~400kb of code, this will add about 10-20ms of CPU-time just to init based on complexity. I did a FIDO2 implementation that load a lot of code and did measurements of the init-time of that code.

Related:

1 Like

And your lodash library, is it installed from package.json, or do you publish it manually in some directory (in this case, how does the files tree look like) ?

You can just do yarn add lodash and then use it like in my example, webpack will package lodash as a dependency then.

For my own code, I use simple relative modules from directories that are imported by webpack.

Example dir /worker/libs/mymodule.js contains:

export async function MyMod () { // code here }

And import with:

const { MyMod } = await import('./libs/mymodule');

Where the init file is in /worker/index.js

If it’s a default module, you’d need to do this for dynamic import:
const { default: MyMod } = await import('./libs/mymodule');

1 Like

I’m not familiar with webpack, but I understand from your messages that webpack should do the trick, still allow to perform dynamic imports, while generating only 1 file (bundled) as build output.

I’ll check the source code of index.js in output, to see how it is generated by webpack.

Spoiler Edit :
Once bundled by webpack, there is no such await import instructions kept in the bundle, because the default webpack configuration outputs ES5 code. Modules are transformed into objects and their exports into functions. It means ALL the code is parsed when loaded in the V8 worker, which is the opposite of the effect searched throught dynamic imports. It means we need something better than the default webpack build.

Whenever possible, I would like to stick with a custom build, without webpack, to keep things under control.

One equivalent ES2020 code could be:

Solution 3 - Text import

// At build time, inject this code
let modules; // Global
const _import = async (s) => {
  // At build time, generate a list { moduleFile: moduleTextContent, ... }
  modules ??= JSON.parse(`{ 
    "my-module-1.mjs": "export const a = 42;", 
    "my-module-2.mjs": "export const b = 84;" 
  }`); // https://v8.dev/blog/cost-of-javascript-2019#json
  if (typeof modules[s] === 'undefined') { return null; }
  if (typeof modules[s] === 'object') { return modules[s]; }
  // Get a temporary local file url blob:
  let blob = new Blob([modules[s]], {type: 'text/javascript'});
  const blobUrl = URL.createObjectURL(blob);
  // Import and cache for next call
  modules[s] = await import(blobUrl);
  // Free memory
  URL.revokeObjectURL(blobUrl);
  blob = null;
  return modules[s];
}
// At build time, replace all "import(" by "_import(" and add .mjs ext
const a = (await _import('my-module-1.mjs')).a;
const b = (await _import('my-module-2.mjs')).b;
console.log(a, b); // 42 84

Text import ensures a JIT parse and compile of the JavaScript, while keeping all modules bundled into one file.

Webpack is unfortunately a requirement, I wouldn’t recommend changing to any other because you’ll get issues with tree-shaking and code replacement for node APIs etc.

And no, you can’t load modules from strings, since it’s blocked in workers.

If build speed is a concern, you can switch to:

That’s the only one that that’s been able to successfully compile my larger worker scripts without issues in production.

1 Like

The above code works in Chrome.

I didn’t tested it yet in CloudFlare workers (V8 isolates).

If you use node_modules dependencies, especially for ES5 code but not limited to, yes, you’d better have a tree shaker somewhere in the build process !

I try to not use external dependencies, to keep things under control.
Partners APIs (ex: Stripe) and the V8 platform (ex: crypto.subtle.*) fulfill my needs.
For now (fingers crossed).

I recently (10 days ago) switched from terser to esbuild. esbuild has proven to be the only builder able to handle latest ES2020 / ESNext instructions and provide a correct minification. It’s an amazing tool, very reactive and friendly to new ES specs !

Glad to hear it is working with your CloudFlare workers.

1 Like

I actually tried to get webpack switched to esbuild in wrangler, but Cloudflare wanted to go with what most businesses use.

1 Like

Well CloudFlare proposes javascript or typescript templates. Having this choice is great. So why not propose different build options as parameters ?

Anyway, the most important thing is:
Alternative choices are possible, by CLI, by config or by code.

1 Like