I could see so many requests from User Agent go-http-client/1.1 in my website. What is this?
You can find more information about this User Agent by searching it on Google.
If you want to block this UA, you can configure an User Agent Blocking rule (exact match only) or a Firewall Rule (exact/partial matches).
Please let me know if I can help you with any further questions.
The go-http-client/1.1 User Agent, is Google’s Bot/Crawler and I think would be a bad idea to block it, unless if you don’t want your site to rank on Google’s Search Engine of course. If that is your case, then you can follow @dmz’s suggestion on How to block it. This article describes
That’s not true. This probably refers to a Go library (developed by Google, according to the source you mentioned, however I couldn’t find reliable information about it).
Even if it was created by Google, it cannot be said to be a Google crawler, at least not based on official documentation:
To those who wish to block requests from
go-http-client UAs while being sure it is not a Google crawler, the solution is to create a Firewall rule like this:
(http.user_agent contains "go-http-client" and not cf.client.bot)
Cloudflare has advanced mechanisms that go beyond the User Agent string and do the verification properly, as described in this other official document:
PS: I have observed this UA (and some variations of it) on several sites that I manage and can confirm that its behavior is not consistent with that of legitimate Google crawlers.
Actually I didn’t suggest anything, I just explained how to block it IF the user wanted to.
This topic was automatically closed after 14 days. New replies are no longer allowed.