Google Crawl Stats Underreported

General Analytics
1

I have noticed for about a month my bot stats for google is much lower (it possibly has happened since google switched to mobile first indexing).

My googlebot is averaging around 3k pages per day from cloudflare stats, msnbot is doing 60k from cloudflare stats. However when I look at webmaster tools it shows gbot doing 80k a day.

2

Cloudflare may not have all Googlebot user agents accounted for, especially any new ones.

If you use Logflare you can see all the individual requests and setup custom filters to show all the googlebots and even make charts of that with Data Studio.