How should I manage Python requests on my sites?

Security
#1

In the last month or so, I have experienced a lot more Python requests appearing in my access logs.

One was so frequent at around 15,000 hits per day that I had to set a firewall rule for it with ASN and user agent to block access.

But there are many others, but less frequent.

I know I can set a firewall rule: (http.user_agent contains “python-requests”)

But is it necessary or even wise to do so?

I’m really not sure what these Python requests are trying to do because I don’t have enough knowledge.

Can anyone help me in finding the correct course of action to take?

#2

I do it. Those python requests are unwanted, in my book. The requests I see look like malicious probes using a tool built on python.

#3

Thanks for your quick reply!

Okay, I’ll keep blocking them as best I can.

2 Likes
#4

Block them.
I suggest looking into blocking malicious ASNs since you will (soon) realize that the attacks often come from the same conflictive networks.

1 Like
#5

Yes, I have quite a few small ASNs covered. But many of these Python hits are coming from Amazon and Microsoft servers. So I can’t block by ASN, only by a user agent.

1 Like
#6

Honestly, because these cloud providers have a free trial, attackers abuse that a lot. By blocking those ASNs, you won’t block legitimate bots if you allow them.

https://developers.cloudflare.com/firewall/known-issues-and-faq#how-does-firewall-rules-handle-traffic-from-known-bots
You can set up a firewall rule with the highest priority that allows cf known bots and expand that rule in the future if you find that CF doesn’t whitelist some bots you find legitimate.

This can’t be implemented in all scenarios; however, I’d give it a try if you can!

1 Like
#7

This topic was automatically closed 3 days after the last reply. New replies are no longer allowed.