2020/2/28 Workers Runtime Release Notes

New from the past two weeks:

  • Fixed a bug in the preview service where the CPU time limiter was overly lenient for the first several requests handled by a newly-started worker. The same bug actually exists in production as well, but we are much more cautious about fixing it there, since doing so might break live sites. If you find your worker now exceeds CPU time limits in preview, then it is likely exceeding time limits in production as well, but only appearing to work because the limits are too lenient for the first few requests. Such workers will eventually fail in production, too (and always have), so it is best to fix the problem in preview before deploying.
  • Major V8 update: 8.0 -> 8.1
  • Minor bug fixes.
3 Likes

Did something else change in preview (Quick edit)?

Because my larger scripts fail to print any console.log output, while the smaller ones still do.

No, that’s unexpected.

Does it happen for both top-level console.log() statements and those in the event handler? And, how big is the script?

It happens script-wide, even as a global.

I tested with a different bundler, just in case it would be a webpack issue. Same result.

Script size, according to wrangler, 193kb.

Hm, this is very odd.

Even if it’s just a very simple script:

addEventListener('fetch', event => {
  event.respondWith(handleRequest(event.request))
})

/**
 * Fetch and log a request
 * @param {Request} request
 */
async function handleRequest(request) {
  console.log('Got request', request)
  return new Response('OK', {
    headers: { 'content-type': 'text/plain' },
  });
}

It will not print any console.log on this specific worker (Size was just a coincidence), but it works as expected if i paste the same into another worker. And yes, I’ve cleared cache and tried different browsers, it happens only on this worker.

Could this be related to routes somehow?
Then why does it happen now, it worked before this last update.

UPDATE: Found another script it happens on, it only has one route /*.

UPDATE2: The rest of the scripts features work just as normal, it’s only console.log that won’t work.

Hi @thomas4, I was able to reproduce the issue, and it does seem like a regression. I rolled back a component that might have been causing it, and can no longer reproduce. Can you confirm?

2 Likes

Thanks @harris, now it works as expected!

Great that you could fix it so fast.

Hi @harris could you have a look at this issue too? This is a very strange behavior, and wondering if has anything to do with this release

@harris: Not sure what you just changed, but now I’m getting script exceeded time limit in the preview after a second of running, even though the script works fine outside of preview, even at multiple requests.

Another regression? I’m 90% certain it worked just a few minutes ago in preview.

Hi @thomas4, could you describe what the script does, and elaborate on “after a second of running”? Does that mean after multiple requests, or a single request that takes one second, or something else?

If you mean “after one or two requests”, then off the top of my head it sounds like the script uses a lot of CPU time and would eventually break on the edge (see the first bullet point in the release notes above).

Alternatively, I suppose that a script which uses > 50ms CPU time on the very first request, but then uses very little for all subsequent requests, would potentially never break on the edge, but exceed the CPU limit in the preview.

Yeah, I just ran a load test and it’s definitely CPU-bound.

I’m using jsrsasign to verify 3 different certificates… That was obviously enough to drain the CPU :cry:

Hmm, does jsrsasign use the WebCrypto API? If not, can you use the WebCrypto API directly, or does it not implement the right algorithm(s)?

I honestly think it would be way too much work to re-implement it and it doesn’t support the algorithms.

I implemented FIDO2 in workers and 100% pass the validation tests yesterday…
And now I cannot use it at scale. Lovely…

Another project to the pile of “cannot run”

I’m sorry to hear that, @thomas4. :frowning:

I respect that it might be more work than it’s worth to switch from jsrsasign to the WebCrypto API, but I’d still be curious to know which algorithms we’re missing. I would expect using the WebCrypto API (i.e., a native implementation) is always going to be the only reasonable way to do RSA signing in a worker, so it’d be great if we supported what people need.

I’m sure native would work, it seems very close, but I don’t have the time any more.

RSASSA PKCSV15 SHA256
RSASSA PKCSV15 SHA1
SECP256R1 ECDSA SHA256
SHA256 RSA
SHA384 RSA
SHA512 RSA
SHA1 RSA
SHA256 RSA MGF1
SHA384 RSA MGF1
SHA512 RSA MGF1

… at a minimum for FIDO2 to work.

UPDATE: Basically, RSA is missing, see list above.

I’m pretty sure these three are supported: Cloudflare Workers

Hmm, these I’m not sure about – do you know if there any alternate names for these? Maybe the first four are RSA-PSS, while the last three (MGF1) are RSA-OAEP?

MGF1 seems to be it’s own thing: Mask generation function - Wikipedia

Update: Did some tests with default minimum, fails on SHA256withRSA. In preview it fails directly, in production worker fails after 20 req.

Hi @thomas4, regarding the crypto algorithms: could you share a source for the list that you posted, or references to algorithm definitions such as RFCs or other standards? My intent is to try and map them to the conventional algorithms defined in the WebCrypto spec; failing that, to see if there is a way to define them in terms of the WebCrypto API.

The list you posted seems to imply that FIDO2 depends on three different RSA algorithms. I recognize RSASSA PKCSV15 SHA* as RSASSA-PKCS1-v1_5. I strongly suspect SHA* RSA MGF1 refers to what WebCrypto calls RSA-PSS because it is referred to in this FIDO2 spec (as RSASSA-PSS), it is the only mention I can find of MGF1 in that document, and it is the only RSA signing algorithm I am aware of that uses MGF1 (RSA-OAEP does, too, but that’s an encryption algorithm, so probably not used by FIDO2). That leaves SHA* RSA as a bit of a mystery to me.

I hope you mean using the pure JS implementation?

If the FIDO2 spec says that it’s RSA-PSS then it probably is, I was using a reference to the older specs when I built that part of the implementation.

Yes, the pure JS based.

I’m currently setting up Rust to get a WASM going that will resolve the RSA verification. I’d guess this is a much faster route than you adding it to Webcrypto API, right?

Got it. In that case I suspect RSA-PSS is the only one we’re missing.

That is probably a safe bet. I’ve filed an internal ticket for RSA-PSS, but I can’t make any prioritization guarantees.

1 Like