Site is fine via browser but 403 accessed programmatically

** I DO NOT OWN OR CONTROL THE WEBSITE **

I am able to access https://unjobs.org/organizations/ctbto using any browser, no problem.
But when I try to WebClient.DownloadData() within my web application, I get 403. I think it is not an origin fault because I see this in the HttpWebResponse:

     Server => "cloudflare"

I don’t expect you to solve my problem; however, I would like to know what I should ask the site owner? I’ve read about “WAF rules”, is that something he controls?

Are you using Cloudflare proxy (orange cloud) and do you have Bot Mode active and set to block or challenge perhaps?

I regret that I do not know what any of those terms mean, nor do I have any idea how their Cloudflare settings are configured. The site belongs to the customer of the web application I maintain, and I need to access a page of that site with my C# code. All I get is 403 and I am hoping to know the best way to ask the client’s site admin when I finally contact him/her. What’s killing me is that requests from any browser on any client machine get a valid response; I’ve traced the headers and they are only sending four headers:

  GET /offices/ctbto_vie HTTP/1.0
  Host: unjobs.org
  Accept: text/html, text/plain, text/css, text/sgml, */*;q=0.01
  Accept-Language: en
  User-Agent: Lynx/2.8.8dev.15 libwww-FM/2.14 SSL-MM/1.4.1 OpenSSL/1.0.1e-fips

which I’ve emulated and it still fails, so I’m at a loss.

Thanks for your time.

In this case only the customer will be able to tell you how it is configured and if Bot Mode is active or not.

1 Like

I bet these two are the triggers if you are getting blocked.
Can you change that somehow to something other and try again?

Does not support JavaScript.

. . . but those are the headers which work in a browser. My code has tried both with and without those headers (and others, especially User-Agent) and always fails.

This topic was automatically closed 15 days after the last reply. New replies are no longer allowed.