Solution to fetch() API escaping characters


#1

I am trying to query ARIN’s REST API for network info, but in the URL there is a ; that gets escaped which makes the API not work, is there any solution to this? Is it an error in the Workers’ implementation, @KentonVarda?

async function API_whois(request, url, URLpath, ip) {
  let whoisIP = URLpath[4]

  let newURL = 'https://whois.arin.net/rest/nets;q=' + whoisIP + '?showDetails=true'
  return fetch(newURL, init)
}

#2

Doh, this indeed looks like a bug in our implementation! @harris is working on a fix. Most likely, the fix would be deployed the first week of April. Sorry for the inconvenience.

It’s weird that this API uses a ; instead of a ? to separate what looks like a query parameter, but we’re definitely breaking spec by escaping it. :confused:


#3

Thanks @harris! Had a couple of minutes of :thinking: when the built URL worked on the browser, but return a 404 on the Worker preview, then I tried logging the request URL from the request itself.

Yeah, I’m not gonna question ARIN, but there is some strange thing going on there… There is a variation which has a slash separating the pieces, but returns less info. Strangely enough, but given the semicolon nothing is strange, they change the response format based on the Accept header…

If you are interested in one of the strangest API implementation I have seen: https://www.arin.net/resources/whoisrws/whois_api.html

PS: a suggestion on the Workers code editor: would it be possible to implement some sort of code collapsing? Because when you get to multiple long functions a reduction of the clutter would help!


#4

I’m on it – thanks for the report! :slight_smile: We’ll let you know once the fix is deployed.

Regarding the code collapsing suggestion, I think once you get to the point you’re describing, it’s best to switch to your-favorite-code-editor (vscode, sublime, vim, emacs, etc.) then use the HTTP API to upload your worker script. Will pass the suggestion on, though – thanks!


#5

Did that, only using copy and paste, didn’t want to use the API to just upload a single file, single developer here…


#6

I have the same problem with an api that contains @ in the url, which is getting escaped.


#7

I have the same problem with an api that contains @ in the url, which is getting escaped.

Thanks, I’ll make sure that gets fixed at the same time. I presume the @ sign is in the URL path segment?


#8

Any update on this @harris? Will this make it out this week?


#9

@erikkri and @matteo, yes, the fix for the URL path escaping bug just went live! Please try it out and let me know how it goes.


#10

I am still seeing the error…

Should be:   https://whois.arin.net/rest/nets;q=8.8.8.8?showDetails=true
It is:       https://whois.arin.net/rest/nets%3Bq%3D8.8.8.8?showDetails=true

#11

@matteo, I’m investigating now.


#12

Thanks!


#13

@matteo, I can reproduce the problem in the preview on the Cloudflare dashboard, but when I save the script to production, it works correctly. Can you confirm that on your end?


#14

Agreed and confirmed!


#15

@matteo We figured out the glitch and the preview will be updated shortly. Sorry about that – and thank you for testing!