Ampersand escaping?

Hello,

I’ve quite a weird issue with the requests going through CF.

For internal usage, I’m performing a request like this:

myAPI/Search/a?alfa=beta&gamma=delta

where, as you can see, the ‘&’ was escaped with its encoded sequence '&'

When I perform the request directly, I find the following URI query parameters in my web server’s logs:

alfa=beta&gamma=delta

On the contrary, when the request is performed through CF, they are:

amp;alfa=beta&gamma=delta

You can see that in the latter request, the URI query parameters are preceded by the encoded form of &.

Unfortunately, this throws an error in the application.

Strangely enough, if my add another parameter:

myAPI/Search/a?alfa=beta&gamma=delta&epsilon=teta

the result is correct.

How can I use the encoded form of & without triggering such behavior?

Feel free to reply if you think the problem is not clear,

thanks,

Marco

Have you enabled Query String Sort? (on the Caching/Configuration page of the dashboard)

What is your URL Normalisation configuration? It should not encode or decode &s, but just in case.

https://developers.cloudflare.com/rules/normalization/manage

Hello Michael,

Query String sort is enabled, but that shouldn’t be a problem as:

  • the request doesn’t depend on the order of the parameters
  • the query is not cached, so that settings shouldn’t affect this case at all.

Indeed Normalize incoming URLs is enabled for the incoming URLs, while URLs to the origin are not.

I’ll have a closer look on how this setting works!

I’ve really appreciated your feedback, thanks!

Try disabling Query String Sort. It is not documented anywhere that I can find, but it makes sense that the query string is disassembled, and then reassembled using the standard (non-encoded) &.

Curious to see if that is the case.

According to the documentation you shared with me, my current settings:

  • Normalize incoming URLs: on
  • Normalize URLs to origin: off

Shouldn’t affect how my webserver receives the request, as URLs should not be normalized towards the origin. My case should be the following:

www.example.com/%68ello On Off www.example.com/hello www.example.com/%68ello

Indeed, it’s related to the “Enable Query String Sort” setting :wink:

Disabling it, the behavior is now correct. :thinking:

I’ll discuss with my colleagues how we should handle this, as this setting can potentially affect the cache effectiveness. Do you have any experience with this? Do you know of an alternative way we could handle this while keeping the query string sort enabled?

very helpful, thanks!

You should not URL encode ‘standard’ ampersands in a query string. In a normal query string they are unambiguous, so should not need to be encoded:

https://www.example.com/wisdom?fruit=tomato&fruit-salad=no

If you have something like this it is a different scenario, and the ambigious characters need to be % encoded as this URL is not clear:

https://www.example.com/redirect?code=301&url=https://www.example.com/wisdom?fruit=tomato&fruit-salad=no

Ampersand encoding of & is really needed when you are embedding a url in HTML:
<a href="https://www.example.com/wisdom?fruit=tomato&amp;fruit-salad=no">

2 Likes

This topic was automatically closed 3 days after the last reply. New replies are no longer allowed.