Elastic dashboard not performing

General Dashboard
,
1

We are a software company that is hosting web software in the cloud. Every instance is behind Cloudflare, this is around 3800 instances atm. So im guessing the issue is too much data.

We have implemented this solution: Elastic · Cloudflare Fundamentals docs

Cloudflare logs are being pushed to an Amazone S3 bucket. With a local logstash installation we pick them up and forward them to Elastic cloud. I can see all the data in the elastic cloud under Discover. We have added all the data from the message part into separate json fields to make it readable and searchable.

So far so good. We have imported the dashboard.json from githhub: Release v0.1 Initial Public Release of Cloudflare / Elastic Integration · cloudflare/cloudflare-elastic · GitHub

And this will give errors failing shards and timeouts.

When I remove the scripted fields the board will load and show me some data. A day later when more logs have been collected it won’t show any data anymore because of failing shards.

I have tested with 10 up to 200 shards but no difference. The shards keep on failing.

Can anyone confirm that we are just dealing with too much data for these dashboards or is there an option for us to tweak it?

2

I’d first take this to the Cloudflare engineer assigned to you. You are likely to get a more concrete answer than here. Enterprise features are rather rare round here, particularly on scale.

3

I logged a call and found out Cloudflare dashboard is made and tested on Elastic 6.x where we are using 7.x which is not yet supported by Cloudflare. Unfortunately no solution for me so we will be working without the dashboard in discover mode

4

I am working on a Filebeat Module for Cloudflare and I could use some help from a potential user, see beats/x-pack/filebeat/module/cloudflare at 2666-cloudflare-module · legoguy1000/beats · GitHub.

5

This topic was automatically closed 3 days after the last reply. New replies are no longer allowed.