What is the name of the modal you’re running?
claude-3-5-sonnet-20241022
What is the error number?
Exact error number: None. After posting, click the error number for fix ideas. (There’s no specific error number since this is a parsing failure, not a thrown error with a code. The Gateway logs simply show incorrect/missing data.)
What is the error message?
Paste the exact error message: None. (Similarly, there’s no error message; the issue is that the Gateway logs show 0 or missing values for input/output tokens and costs, despite the Anthropic API response containing the data.)
What is the issue or error you’re encountering
Cloudflare AI Gateway is not parsing token usage data (input_tokens and output_tokens) from Anthropic API responses, resulting in inaccurate or missing token counts and cost estimates in the dashboard logs. Despite Anthropic’s API returning a usage object with these fields, the Gateway logs do not reflect the values, which impacts observability and cost tracking. For example, an Anthropic response includes: json Collapse Wrap Copy { “type”: “message”, “role”: “assistant”, “model”: “claude-3-5-sonnet-20241022”, “stop_reason”: “end_turn”, “stop_sequence”: null, “usage”: { “input_tokens”: 4144, “cache_creation_input_tokens”: 0, “cache_read_input_tokens”: 0, “output_tokens”: 475 } } The Gateway logs should show 4144 input tokens, 475 output tokens, and an estimated cost of ~$0.0196 (based on $3/million input tokens and $15/million output tokens for Claude 3.5 Sonnet). Instead, the logs show 0 or missing values for both tokens and cost. This issue appears specific to Anthropic’s integration; OpenAI requests through the same Gateway parse tokens correctly.
What steps have you taken to resolve the issue?
Steps taken to resolve:
Confirmed the request is routed through the Gateway using the correct URL: https://gateway.ai.cloudflare.com/v1/<account_id>/<gateway_id>/anthropic.
Tested with a simplified payload ({“model”: “claude-3-5-sonnet-20241022”, “messages”: [{“role”: “user”, “content”: “Hi”}]}) to rule out payload-specific issues—same result.
Compared with OpenAI requests, which parse correctly, isolating the issue to Anthropic.
Reviewed Cloudflare AI Gateway documentation but found no mention of specific parsing limitations with Anthropic.
As a workaround, I’m manually extracting input_tokens and output_tokens from the Anthropic response and calculating costs (e.g., $0.019557 for the example above).
What are the steps to reproduce the issue?
Steps to reproduce:
Set up a Cloudflare AI Gateway instance with an Anthropic endpoint (e.g., https://gateway.ai.cloudflare.com/v1/<account_id>/<gateway_id>/anthropic).
Send a POST request to the Anthropic API through the Gateway with a valid Anthropic API token:
bash
Collapse
Wrap
Copy
curl https://gateway.ai.cloudflare.com/v1/<account_id>/<gateway_id>/anthropic -X POST
–header ‘Content-Type: application/json’
–header ‘Authorization: Bearer <anthropic_api_token>’
–data ‘{“model”: “claude-3-5-sonnet-20241022”, “messages”: [{“role”: “user”, “content”: “Test prompt”}]}’
Receive a response from Anthropic containing a usage object (e.g., input_tokens: 4144, output_tokens: 475).
Check the Cloudflare AI Gateway dashboard (AI > AI Gateway > Logs) for the request. The logs should show the token counts and estimated cost but instead show 0 or missing values.