Type
Product improvement
Description
AI Gateway filter support filter by metadata
Benefit
I am trying the Evaluation feature in the AI Gateway. When I create the dataset, it only allows filter by status or duration. I feel it may not be beneficial to evaluate prompt changes.
For example, I set a metadata { "prompt_version": "2024-10-05" }
and I can filter by “prompt_version” to create a dataset that uses the same prompt template. It will be more helpful to compare different prompts or create A/B testing.