First, I don’t think I rise to the level of novice. My apologies in advance.
I had been successfully communicating with a FlightRadar web site using my IoT device for many weeks. All of a sudden, I’m now getting the 403 error. Nothing changed on my side. Looking into debug statements over WiFi on my IoT, I see the statements below which indicate to me that the SSL certificate is valid.
Since this is an IoT device, the certificates are installed manually. Of course, there are no caches.
But then I get the 403 error, response also shown below.
As I said, this has worked perfectly until a few days ago. The IP address does point to FR24 and the response does say the server is Cloudflare.
I’m lost and don’t even know where to go to find this. Any help please?
Can you try the link in your browser and see if you get a block message, or a challenge?
You will need to contact FR24 and give them the ray ID. They will be able to check their logs to see why your request is being blocked/challenged. They set the rules for access to their site, not Cloudflare.
FlightRadar has a playground for beta users and I can successfully talk to their server in the playground.
The playground also shows examples in various formats. I’m using a GET in my IoT, but from a shell on a Linux machine at my domain I can get a response using an example curl command.
That does indicate it’s specifically my IoT, but the debug I posted makes it seem the certificates are OK. (I also tried several devices just to rule that out.)
What is a ray ID? (FlightRadar is often slow in support, probably due to a half day time difference between them and me.)
OK, took some time… FR24 confirms that my IP is not being blocked. What is strange is I previously had difficulty with a certificate and did not get the 403 error.
My IoT doesn’t like the new GTS certificate so that appears to be the problem.
It’s been almost 2 months that I’ve been chasing this error. I think I’m homing in on the base problem and it has something to do with Cloudflare.
Previously, I would resolve the url to an IP address and everything worked fine. Then it stopped working. Resolving the url to IP has not changed and I get an ipv4 of 104.18.97.112. Doing a whois on this I find this is a Cloudflare IP address. Pinging the url, I get the ipv6 of 2606:4700::6812:6170. A whois says this is also Cloudflare.
I’m not part of FR24 so I don’t know how traffic between their servers and my IoT is handled by Cloudflare. I don’t know if this is an FR24 server or a Cloudflare server handling FR24 traffic.
I arranged a debug session with the FR24 development team. In over 30 minutes, I sent about 2000 messages, each one returned the 403 Forbidden error as shown in my first post. And, the FR24 team said they saw no traffic from my IP.
As of now, it does appear to me that Cloudflare is blocking my traffic. I’m guessing this is because of the IP address issue described above.
sjr asked what the ray ID is. FR24 asked me the same question. I have no more information other than what’s in my first post. But the statement CF-RAY: 896dd1f20f86f963-SJC appears to me to answer that question. Remember, this is an IoT. The traffic is not generated by a browser running in an O/S. So I’m not sure that their is a ray ID.
If you were getting a Cloudflare 403 challenge/block message, then they won’t be seen on their origin.
Every request has a ray ID. It will be in the header of every request arriving at an origin and response returned from Cloudflare.
The fastest solution is for FR24 to look up some ray IDs in their event logs.
Cloudflare won’t be blocking you, but settings in FR24’s Cloudflare account may well be.
To check, you can get your device to make requests against my sites that are behind Cloudflare… https://icfbm.net/ (for IPv6 or IPv4) https://icfbm.com/ (for IPv4 only)
(both will issue a redirect response if you just query the home page from your IoT device)
I’m not smart enough on this topic to figure out what files you might have at that domain, but if I do a GET to https://icfbm.com or https://icfbm.com/index.htm(l) or just index.htm I get no response. I am opening a secure socket successfully to 143.244.220.150 though.
Don’t follow the redirect, it redirects to another site outside Cloudflare. But if you are able to cnnect and receive the redirect header, it means Cloudflare isn’t blocking you.
I cannot get a response at all from icfbm.com, but I do get responses from gmail.com, googlemapsapi.com, my own web site, and fr24, but only the forbidden.
I am on the road this week and will have to put this aside until the following week.
I think I’ve narrowed the problem. I resolve the URL of the API to an IP address. Doing a whois on that address returns Cloudflare. FR24 says they use a reverse proxy at Cloudflare to access their server. (Not sure what that is or how it works.)
So I’m guessing:
The resolve function I’m using to get the IP address is not working properly.
or, Cloudflare is not properly forwarding my messages through their servers using the IP address I have.
FR24 has never (since it broke) seen messages from my IP address so they have no ray IDs that can compare.
The problem appears to be on one side or the other, with me in the middle.
It means that when you send a request to their domain, it is actually sent to Cloudflare instead. FR24 can set Rules on Cloudflare to decide whether a request is forwarded to their server or blocked by Cloudflare’s firewall.
They would not see the request if it was blocked, that is why you would need to supply them with the RayID so they can check their Cloudflare logs for that RayID.
The RayID is part of the headers that you receive with the 403 HTTP response, and you have posted it in your opening post:
Security logs on Cloudflare aren’t kept for very long, so after you make a request and receive a RayID, you need to contact FR24 relatively quickly and have them check their Cloudflare security logs for that Ray. The will will tell them why your request was blocked.
Unfortunately, the previous ticket on this was resolved (?) and closed before the investigation was complete.
The IP address and ray ID data was provided to FR24 and they saw no evidence that the traffic was getting through. All indications are Cloudflare is blocking the traffic. This could possibly be because Cloudflare has changed to a 384 bit ECC which is not supported on my side.
Here is the debug information from my IoT. I’ve been told this indicates SSL decryption is successful, but it appears to be not so.
Was the site working with SSL prior to adding it to Cloudflare?
Yes
What is the current SSL/TLS setting?
Full (strict)
What are the steps to reproduce the issue?
Not sure what the question above means.
The previous solution (?) stated FR24 needed to search the logs using the ray IDs to find out why the traffic was blocked. FR24 found no evidence of any traffic.
Thanks Laudian MVP '24. I have passed on your comments. I am in California. My contact is in Bulgaria. Their tech team is in Sweden. Two days to get an answer of any kind (not counting back-and-forth) is not sufficient to resolve this.
“I am unsure of Laudian’s credentials, however, I fully trust our engineering team and their findings.”
So I am the proverbial stuck between a rock and a hard place. I don’t have the access to the Cloudflare logs as my company is not the Cloudflare user. I cannot generate new ray IDs (as of now) since FR24 has shut down their beta playground. FR24 appears to be certain that my IP is not being blocked even though they did not see any traffic from my IP when I was successfully using their playground to communicate with their server.
I have a year and a half invested in this and prefer to not waste that effort, but honestly do not know what else to do.
If FR24 is not willing or able to help you, then I’m afraid there is really not much left.
I’ve enabled all of Cloudflare’s standard rules on test.laudian.de - if you want, you can make a request to that and see if you are blocked as well. If you are, share the Ray ID and I could have a look at why you were blocked.
Might take a few requests so I can change settings in between.
You’ve graciously offered your help and that is much appreciated. This suggestion is similar in format to a previous one you’ve made but I’ve not been able to make it work. Talking to several different web sites in similar ways, I’ve tried: