We use logpush, and ingest those logs into BigQuery following this guides;
I do query the count and SUM of CacheResponseBytes and EdgeResponseBytes in a day, but those results are not the same as Cloudflare Dashboard.
In the dashboard last 24h around 2020-05-15 21:00 JST
51M Total Requests
27.31TB Total Bandwidth
With BigQuery on 2020-05-14
79M records (count)
54TB SUM(CacheResponseBytes) or SUM(EdgeResponseBytes)
SELECT COUNT(*) AS requests, SUM(CacheResponseBytes) AS CacheResponseBytes, SUM(EdgeResponseBytes) AS EdgeResponseBytes, FROM `my-project.cloudflare_logs.my-domain` WHERE DATE(EdgeStartTimestamp) = "2020-05-14"
I expect differences in few percentages or so, but Bigquery result bandwidth is almost x2 and request are x1.5.
I know the timespans are different, but last week analytics is almost flat.
How can I get almost same results from dashboard and BigQuery (logpush)?