[BUG] Inconsistent URL behaviour

The implementation of URL in workers is inconsistent with browser implementations.

Top is CF Workers, bottom is Chrome 75.
01%20AM

Notice that the query string is decoded but not encoded.

Additionally, URL is missing the forEach method, and fails to pick up query parameters

> new URL('https://google.com/?term=something%20something%20something').searchParams.forEach(console.log)
> VM170:1 Uncaught TypeError: (intermediate value).searchParams.forEach is not a function
    at <anonymous>:1:84
> Object.keys(new URL('https://google.com/?term=something%20something%20something').searchParams)
> []
> Object.keys(new URL('https://google.com/?term=something+something+something').searchParams)
> []

URLSearchParams cant be passed to Object.keys().

The forEach method is actually missing.

And as far as the encoding is concerned, it would seem as if Cloudflare actually encoded it correctly, as spaces in the query string should be encoded with a plus sign

Right - my bad about the Object.keys. You can achieve what I’m looking for via [ ...searchParams].forEach((key, value) => ...)

The concern is that it’s inconsistent with everything else.

This particular issue comes up when using WP sites which will 301 you from URLs with +s in the query to the encoded URL.

As far as I understood the pluses should be actually correct. Where did you experience an issue with them?

https://wordpress.org/?something=search+term will auto redirect to https://wordpress.org/?something=search%20term

And why is that an issue?

Consider this flow:

  1. Someone comes in on https://wordpress.org/?soemthing=search%20term
    1.1 it doesn’t matter if they come in on https://wordpress.org/?soemthing=search%20term or https://wordpress.org/?soemthing=search+term
  2. You process that URL in the worker, which will produce https://wordpress.org/?something=search+term
  3. A subrequest is made to https://wordpress.org/?something=search+term
  4. WP 301s you to https://wordpress.org/?something=search%20term
  5. Go to Step 1

Hi @tyler.sebastian, you’re absolutely correct: our URL class implementation is buggy, and this is a longstanding problem. It’s unfortunately difficult to correct, because fixing it risks breaking working scripts that rely on the buggy behaviors. We need to first come up with a migration path.

In the interim, you could polyfill a conformant pure JS URL class implementation, though I realize this is not an ideal solution.

Harris

Thanks for the response, Harris. We’ve polyfilled with whatwg-url.