Capture HTTP traffic in Graphana?

I am looking at the Cloudflare Grafana plugin, but it seems to only support DNS metics. Is there a way to also expose HTTP traffic metrics via Grafana?

Those metrics are only available to users running Cloudflare’s Enterprise plan, where they are pushed to a third party source of your liking. On other plans there are no ways and there is no direct API to do that automatically, as far as I know.

3 Likes

Thanks for the info, we are on an enterprise plan. Do you happen to know where this is configured in the Cloudflare dashboard? (I’m still very new to the system!)

check out logpush documentation at https://developers.cloudflare.com/logs/logpush

2 Likes

Yeah, also you can contact your CSM Engineer, he will be able to help :slight_smile:

1 Like

Apologies in advance, if this is doesn’t quite fit your needs,

however, one helpful thing I link I like to provide if you are not using an Enterprise Plan subscription is this blog article on how to implement Edge-Side Logs from a CF Worker script:

This will give you access to a number of fields and request properties that you can then point at a third-party/external logging endpoint (the example in the post is for an Elastic-Logstash-Kibana ELK Stack but you could utilize for Grafana too).

It may require a little development work on your side so that it fits your specific use-case, but it will allow you to do a close-approximation of Cloudflare Logs without needing an Enterprise sub.

Hope this helps!

4 Likes

Hi, just saw this when searching for other info.

To echo @Peter-CF above, my team is currently logging from the edge into Grafana.

problem

We were trying to log from a worker specifically and track internal worker metrics in grafana but there’d be no difference in what we’re doing and running a worker on a route whose sole purpose it is to log.

what we did

We spun up a service that could collect data from workers and present a scrapeable page for grafana/prometheus: https://github.com/weaveworks/prom-aggregation-gateway. It’s different form the regular Prometheus pushgateway in that the weaveworks service maintains its own state, which is key because state sharing across workers for these counters seemed impractical. Having to run this sidecar service isn’t ideal, but we had to have this state somewhere.

After having the little gateway service up, we just used fetch inside workers to send metrics form individual worker invocations. So even when you are reading out of CF’s cache and not reaching your origin server you still have your programmable worker there to ship out the logs.

2 Likes

I’ll put my $0.02 in here …

You can setup Logflare and then use the BigQuery graphana plugin to chart any SQL query form the BigQuery table Logflare sends your data to.

This gives you the solution @Peter-CF provides with zero dev work.