How to collect cloudflare zero trust log with API in free plan

Zero Trust Gateway
1

Hello, we are testing to introduce Cloudflare VPN.

I am satisfied in all aspects, but it is disappointing that the degree of freedom in log collection seems to be reduced.
(I know that if i use the enterprise plan, can store logs using the logpush and logpull features)

While researching other methods, I found that there was a way to check logs through the API, although it was not indicated in the official API document.

API URL that retrieves logs through login session authentication on the dashboard
(https://dash.cloudflare.com/api/v4/accounts/{identity}/gateway-analytics/activities/l4)

API URL that retrieves logs using API Key
(https://api.cloudflare.com/client/v4/accounts/{identity}/gateway-analytics/activities/l4)

In the case of the dashboard API response, all information is sufficient, but because session information is required, it is unsuitable for an automated log collection system.

And, in the log information retrieved through api.cloudflare.com, information such as user_id, device_id, and source_ip are identified as empty strings.

As a result, I am wondering if there is a way to directly collect logs with all the information using the API.
I would like to directly compensate for the only drawback of the free plan by implementing a log collection program.

thank you

(Log example from api.cloudflare.com)

{
  "result": {
    "time": 1699429412,
    "logs": [
      {
        "session_id": "{my-session-id}",
        "datetime": 1699429333,
        "account_id": "{my-account-id}",
        "user_id": "",
        "device_id": "",
        "virtual_network_id": "{my-vn-id}",
        "rule_id": "",
        "action": 4,
        "action_name": "allow",
        "source_ip": "",
        "source_internal_ip": "",
        "source_port": 62494,
        "destination_ip": "**.**.**.**",
        "destination_port": 443,
        "override_ip": "::",
        "override_port": 0,
        "transport": "tcp",
        "email": "",
        "sni": "****.****.***",
        "last_authenticated_at": 0,
        "src_country": "**",
        "dst_country": "**",
        "proxy_endpoint": "",
        "detected_protocol": ""
      }
    ]
  },
  "success": true,
  "errors": [],
  "messages": []
}
3 Likes
4

First of all, thank you for posting this because it helped me find the relevant API endpoint to pull the logs, which wasn’t easy to find elsewhere.

Turns out that you can get the user and device identity information (including email and user_id) on the free account by adding “Account > Zero Trust: PII” permission to the API token used to invoke the same endpoint you posted. Below is the token configuration we used to get this working:

image
image910Ă—178 12.3 KB

Hope this helps!

2 Likes
6

Thx for the relevant API endpoint. you can not find it anywhere else. With that said. Do you guys know how to filter the logs you get for example this works:

https://api.cloudflare.com/client/v4/accounts/{account_id}/gateway-analytics/activities?query=youtube.com

But if I use any other option then it does not take in account the filter and just shows the latest logs. I have tried several other methods but I cant seem to get it to work. Do you guys know how to solve this issue i’m having? The datapoint i need to filter on is blocked so it should be like this:

https://api.cloudflare.com/client/v4/accounts/{account_id}/gateway-analytics/activities?blocked=true