Searching in radar /verified-bots

if we go to and search for bot names containing “crawl”
we get googlebot and bingbot in the results. why ?
isnt this search meant to be for user agent names ?
if yes, then “googlebot” does not contain “crawl”

I am concerned because sometimes i block (lower(http.user_agent) contains “crawl”). need to make sure that it wont affect google or bing

You’re searching across all of the fields. If you search “monitoring” or “feed”, you get results as well from that category. You can search “Google” to find all of the google bots/search by the owner field.

You can find the user agents of google/bing on their respective pages: Google Crawler (User Agent) Overview | Google Search Central  |  Documentation  |  Google for Developers, and confirm that they do not contain the word crawl.
That’s an interesting rule, probably not too effective since actual bad bots wouldn’t be nice enough to identify themselves like that, but you could just exclude verified bots (not and http.user_agent contains "crawl")

1 Like

I already use not but want to filter down the list

So, do you mean we cant search for bots names containing for example “crawl” in this link

You can. I’m just saying the actual search is checking if the search query is contained within any of the visible fields. Your “crawl” search is matching the values in the Category field, hence why they’re being shown


This topic was automatically closed 3 days after the last reply. New replies are no longer allowed.