API returns imprecise data about uncached requests

Hey there!
I’ve noticed something odd in the Analytics data I get throught the API.

I have several domains (more than 500) and I use the API periodically to get the count of uncached requests and the amount of uncached transferred bytes. I do this in order to know how much traffic actually hit my server. But I’ve noticed that the values I get from the API are irregular and don’t correspond with what I see in my server logs.

Let me explain a bit more. I took a domain and parsed my Nginx logs to get a report that shows the count of requests per day that hit that domain during the last month (values are pretty much the same every day). In order to compare, I used the Cloudflare GraphQL API to get that same information about that domain: I got the exact same values that I got from my Nginx logs, except for some days in which Cloudflare informs way less requests.

Moreover, that unnacuracy presents in a regular pattern, as you can see in this screen capture of the Cloudflare Analytics dashboard:


Notice that it follows this pattern: 2 days ok, low value, 3 days ok, low value, and so on.

And again, two important things:

  1. This happens for all my domains (more than 500), the exact same days and the exact same unnacuracy pattern.
  2. Except for those valleys days, the data I get from my server logs corresponds with the data I get from Cloudflare Analytics.

The problem appears to happen since May this year. That month I changed my server so I had to change the IPv4 for all the zones in my Cloudflare account. Could this have something to do with this inexact data? Why is Cloudflare Analytics showing this discrepancy of data?

Thanks in advance!
Let me know if I can help with anything to get to the bottom of this issue.

Warm regards,
Lisandro

Are you making it so only requests can reach your site through Cloudflare? If you haven’t then it means users can directly hit your site without going through Cloudflare.

There does indeed appear to be a pattern. It’s unlear that the pattern is inaccurate. A single screenshot with no real context isn’t going to be enough to determine that there’'s an error with Cloudflare. Simplest answer is Cloudflare is showing accurate data and there is no discrepancy.

Thank you for your time! I understand what you say. My first thought was that I was doing something wrong. Maybe I am. Allow me to add some more information.

Important note: all the traffic goes through Cloudflare. There is no way to hit the server without going through Cloudflare.

For a quick analysis, I took the top five domains in my account with most traffic. I’ve parsed my nginx logs for those domains to get the total count of requests per day for the last days. What I see is that the total count per day is pretty much stable for every domain. And those numbers are the same informed by Cloudflare, except for those days where Cloudflare reports a very low number.

Here you have a few examples regarding total uncached requests per day:

------- zone: informatesalta.com.ar -------
date         myserver         cloudflare
28/08         173583           177300
29/08         184772           182110
30/08         180540            30140

If I get more days in the past from my server logs, the values I get for this domain are always around ~180k. The traffic that hits the server is pretty much stable, always between those numbers. There is a very small difference between my data and Cloudflare because I take from 0pm to 0pm and Cloudflare does it from 9pm to 9pm. But the values are always around those numbers. However, notice that Cloudflare reports around 30k in this weird pattern:

Here you have a few more cases:

------- zone: lmdiario.com.ar -------
date         myserver         cloudflare
28/08         140551            141860
29/08         163399            154870
30/08         207822             42700

------- zone: ariesonline.com.ar -------
date         myserver         cloudflare
28/08         128533           124290
29/08         128051           126030
30/08         127917            74970

------- zone: diariolaopinion.com.ar -------
date         myserver         cloudflare
28/08          89435            89180
29/08          89282            87330
30/08         101902            23540

This started happening a few months ago, right after I migrated to a new VPS and had to change the IPv4 for every A record for all my domains at Cloudflare (that is, www and @ records).
The traffic that hits my server kept being very stable and predictable as before, but the graphs at Cloudflare started reporting those weird valleys.

What do you think?
Again, thank you very much for your time.
Best regards,
Lisandro

Hello, I hope you’re doing well. I was wondering if you had a chance to review the latest details I posted regarding this issue. Your input would be greatly appreciated. Thank you!

I’ve received an answer from the Cloudflare support team, they explained to me that the traffic through Analytics for non Enterprise accounts are sampled:
https://developers.cloudflare.com/analytics/graphql-api/sampling/

Data sets from sampled logs cannot guarantee accuracy to a large degree as it is sampled. The amount sampled is adaptive and will not be as accurate as your own server analytics. Please check Cloudflare Logs on Cloudflare Enterprise (Premium) Support account for greater sampling (accuracy) potential.
Please note this will only be on LogPush.

This topic was automatically closed 3 days after the last reply. New replies are no longer allowed.