Better builtin log analytics & more logpush options

The dashboard builtin analytics are mediocre and very basic in my opinion. The only way to get the most out of the log data is to use logpush to cloud and use a 3rd party tool like Google data studio to display the data or feed logs to Splunk, Qradar etc. Our team’s job is to get customers on board with the service in a test type of environment so they can see the benefits of it and want to buy in, but we would not be using Qradar, Splunk or other service during this test period. So it’s imperative we have the best analytical tools in place to show results and as fast as possible. With the http logs only being saved 3-7 days max online, logpush becomes our only option. We tested 3 of the 4 logpush cloud options and add-ons to display data graphics from results, and found them to be lacking detailed specifics on how to setup, and it took some time to figure out why data was not being displayed. We used the cloud trial free service for testing and this works fine for now, but when we have to pay for it this becomes more of a challenge to work it in as an additional add-on cost vs if it was already part of a Cloudflare service package offering. Plus it’s more work to maintain logpush setup, and requires additional resources to share results with the customer.

My thoughts on improvements.

  1. Would be ideal if Cloudflare permitted more data to saved online, say 30 days max and improved the builtin graphs to match what some of the 3rd party log analytic tools provide, like an overview of all http log activity (one page shows all), and then break it down by analytic type.
  2. Allow more cloud providers for logpush
  3. Cloudflare creates it’s own cloud service as an logpush option. With a simple click of a button the logs get automatically pushed directly to the cloud, and the dashboard analytics pull data directly from it to create and show log graph and reporting data on the fly and as far back as the customer wants.

I’d recommend you share this with your customer success manager as well; while these forums are a good was to gauge community interest, your success manager/the success email will give you a better chance of this being discussed and triaged (and most customers here don’t even have access to logpush). I personally agree that some managed CF-ran log service would be nice but I also would be fine if just more providers were added like Datadog [I have filled out the provider feedback google form].

As for the analytics data, some metrics are available for 1 year
https://developers.cloudflare.com/analytics/graphql-api/limits/.

Wondering what analytics you would like Cloudflare to provide.

That’s certainly true. You will always manage to get more out of a high quality purpose built tool like Splunk capable of ingesting logs from multiple sources and designed to provide customized views/dashboards.

The need to use a 3rd party log tool is not something I have encountered to demonstrate the value of our platform. Occasionally I’ll run a tool like Catchpoint if we need look at certain performance related items on a Global scale but I suppose every environment is different.

How much extra would you be willing to pay for the service? Would an additional $3k a month for up to 100GB of storage and 60 days of reporting be reasonable? Would pre-built dashboards be acceptable in that scenario or would ad-hoc queries be required? How much of a premium would you pay for ad-hoc query capabilities?

2 Likes

But do appreciate the feedback on additional push options along with the analytics overall. We’ve been making a large investment in improving the analytics overall and recently released improved cache analytics and org level reporting as part of that effort.

1 Like

Thanks

Thanks for your feedback. I am new to Cloudflare, but what I have seen so far I am very impressed with the overall product and service and what it can do. The only thing I see that needs some extra tlc is the analytics for customers that would not be using SIEM. Most larger shops use SIEM, it’s the only way to manage the data, but smaller shops with only a few domains may not want SIEM and the extra costs that go along with that. Even the larger shops may want the advantage of being able to pull up a complete list of on the fly analytics (like what Google Data studio provides using logpush), just to manually see what the environment looks like at any given time. Would also come in handy if there was an investigation or audit and custom reports to pull log data are required. Auditors can ask for and expect data to be at hand for any given period of time, could be 3 months or even as far back as 3 years.

Would an additional $3k a month for up to 100GB of storage and 60 days of reporting be reasonable?
I think that would be in the ballpark when you compare it to the extra work and costs of using an external cloud storage provider with logpush.
Would pre-built dashboards be acceptable in that scenario or would ad-hoc queries be required? If the prebuilt dashboard had the same as what you see in (as example) in what Google data studio offers, I think that would be great and would be all we needed. How much of a premium would you pay for ad-hoc query capabilities? I don’t work in the offering management area, I am on the technical support side of the house. But my 2 cents, I think it depends how many domains a customer has under your service, and could that be part of a ‘package deal’ you offer them. If they can can be shown what this does, and the benefits, I think they would buy in if it’s a reasonable extra cost
Thanks!