Security issue invalid traffic

Hi,

Your help will be greatly appreciated.
I have WP Multisite but is only one site. Is an informational site with ads.

  1. I have to integrate with Ezoic in my server. Cloudways implemented X-Middleton by advice of Ezoic for this purpose, besides that is not working because I cant see user IP, the consequence is that I cant block with firewall rules bots with X-Middleton in the user agent.

  2. My Ezoic account have been paused due to invalid traffic. I don’t know which is the source of the invalid traffic. But they pointed that I have to ask support from Cloudways, In Cloudways they told me to look for help in Cloudflare, They offer me to restrict the traffic to the one is camming from Clouflare, but they are not doing it. How can I scan or audit what is going on?

  3. I have Cloudflare firewall rules, and a few days ago I enabled Under Attack and Bot Fight. But wasn’t enough.

  4. Cloudways Bot protection app is incompatible with Ezoic, so they recommend me to use Ninja firewall. I had enabled Full WAF mode.

  5. The only things I see in my site:

5.1) Errors in my error log, and I don’t know how to stop this kind of spam.
Ex:
[Thu Jan 14 12:23:45.633013 2021] [proxy_fcgi:error] [pid 32339:tid 140589848721152] [client 3.238.165.133:17040] AH01071: Got error 'PHP message: Error Illegal mix of collations (utf8_general_ci,IMPLICIT) and (utf8mb4_unicode_520_ci,COERCIBLE) for operation 'like' de la base de datos de WordPress para la consulta SELECT SQL_CALC_FOUND_ROWS ar3_2_posts.ID FROM ar3_2_posts WHERE 1=1 AND (((ar3_2_posts.post_title LIKE '% Billige viagra original kaufen schweiz.\xf0\x9f\x8d\xbe\xf0\x9f\x90\xb8 www.DoctorFox.store \xf0\x9f\x90\xb8\xf0\x9f\x8d\xbe Billig glucophage 500 ohne rezept kaufen.%') OR (ar3_2_posts.post_excerpt LIKE '% Billige viagra original kaufen schweiz.\xf0\x9f\x8d\xbe\xf0\x9f\x90\xb8 www.DoctorFox.store \xf0\x9f\x90\xb8\xf0\x9f\x8d\xbe Billig glucophage 500 ohne rezept kaufen.%') OR (ar3_2_posts.post_content LIKE '% Billige viagra original kaufen schweiz.\xf0\x9f\x8d\xbe\xf0\x9f\x90\xb8 www.DoctorFox.store \xf0\x9f\x90\xb8\xf0\x9f\x8d\xbe Billig glucophage 500 ohne rezept kaufen.%'))) AND (ar3_2_posts.post_password = '') AND ar3_2_posts.post_type IN ('post', 'page', 'attachment') AND (ar3_2_posts.post_status = 'publish') ORDER BY (CASE WHEN ar3_2_posts.post_title LIKE '% Billige viagra original kaufen schweiz.\xf0\x9f\x8d\xbe\xf0\x9f\x90\xb8 www.DoctorFox.store \xf0\x9f\x90\xb8\xf0\x9f\x8d\xbe Billig g...'

5.2) I see spam search from well-known bots:
216.244.66.241 - - [14/Jan/2021:12:09:32 +0000] "GET /es/?s= %20apotek%20online%20kodein%F0%9F%A7%B8%F0%9F%8E%8F%20www.Ma yoClinic.store%20%F0%9F%8E%8F%F0%9F%A7%B8%20viagra%20piller% 20uden%20recept%20best%C3%A4ll%20viagra%20p%C3%A5%20n%C3%A4t et HTTP/1.1" 200 12426 "-" "Mozilla/5.0 (compatible; DotBot/ 1.1; http://www.opensiteexplorer.org/dotbot, [email protected]) X -Middleton/1"

Can you enlighten me?
I am struggling I fill that I am hunting in the dark

@irene You can use the WAF and Firewall rules to target that traffic. Depending on the source it may be an ASN or Country that you don’t expect traffic, otherwise you can challenge or block based on the User-agent or any other pattern on the request.


For more dynamic solution we offer premium service upgrades:

Rate limiting

Or Bot Management

Let us know if you have any further questions.

The free tools where not enough, Ezoic scanned the site and said that still see invalid traffic.

The rate limiting, has a very limited configurations settings. I cant find how to setup different options between google bot and other users.

I have a simple informational site, and it is just me, a site owner. The premium tools are out of my scope. And without ads my site is inviable

We exclude GoogleBot from Rate Limiting by default. If you want to bypass RateLimiting for a Bot that we don’t already exclude you can use Bypass action for rate-limiting w/ Firewall Rules.

1 Like

Hi,
Sorry for so many questions but I found the following:

  1. I had set up a firewall rule for block bad referrals traffic, and all my traffic with real users was blocked. Why?

(http.referer contains “concerns.sportshouse.com.ph”) or (http.referer contains “subglobal.net”) or (http.referer contains “unm.org.ua”) or (http.referer contains “pinballspares.com.au”) or (http.referer contains “emmcforum.com”) or (http.referer contains “wallpaper144-781ed.web.app”) or (http.referer contains “ucapanbagus.web.app”) or (http.referer contains “semogalekasi.web.app”) or (http.referer contains “robuxgenerator2018.web.app”) or (http.referer contains “robloxjailbreakhackgenerator.web.app”) or (http.referer contains “breakingnewstrend.web.app”) or (http.referer contains “quizzical-boyd-79e1b7.netlify.app”) or (http.referer contains “affectionate-cori-3dc1f3.web.app”) or (http.referer contains “affectionate-cori-3dc1f3.netlify.app”) or (http.referer contains “bdmedicin.info”) or (http.referer contains “fseriesfanatics.com”) or (http.referer contains “cmaxfanatics.com”) or (http.referer contains “para.inria.fr”) or (http.referer contains “forums.subglobal.net”) or (http.referer contains “forum.unm.org.ua”) or (http.referer contains “pinballspares.com.au”) or (http.referer contains “”) or (http.referer contains “donia2link.xyz”) or (http.referer contains “saldogratispoker.com”) or (http.referer contains “spinbotstudio.fr”) or (http.referer contains “dubaiescorts24forum.com”) or (http.referer contains “palais.beesims.com”) or (http.referer contains “primalcarnageforums.com”) or (http.referer contains “www.e-tahmin.com”) or (http.referer contains “brodzio.pl”) or (http.referer contains “euvapor.com”) or (http.referer contains “movietato.com”) or (http.referer contains “phwow.sk6.ru”) or (http.referer contains “plbm.eu”) or (http.referer contains “forum.3dnatives.com”) or (http.referer contains “edgefanatics.com”)

  1. I was checking my 404 file, so I requested a non-existent file, and I found the request coming from a bot. Why?
    Mozilla/5.0 (compatible; GrapeshotCrawler/2.0; +http://www.grapeshot.co.uk/crawler.php)

  1. Without seeing the Firewall Event Log entry for that denied request, we could only speculate. My speculation is that the Referer contains “”.

  2. It says Grapeshot. That’s on “The List”:
    https://developers.cloudflare.com/firewall/known-issues-and-faq

  1. An expression built using the Visual Expression Editor in the Firewall Rules UI does not require you to manually escape those special characters.

I used the visual expression editor. I didn’t add characters manually

  1. The question is: Is Cloudflare confusing my IP which is whitelisted, whith GrapeshotCrawler?

I don’t see how that’s related to the allowed crawl by Grapeshot (a Verified Bot). The request for ‘nada’ was executed by Grapeshot.

In any way can be coincident.

This is from my access log from today
54.237.155.65 - [26/Jan/2021:13:35:52 +0000] “GET /index.php” 404 0 - 24134 21886 0.203 23068672 69.02% 4.93% “/es/nada”
3.219.31.158 - [26/Jan/2021:14:24:51 +0000] “GET /index.php” 404 0 - 24134 23953 0.203 23068672 59.16% 9.86% “/es/nada/nada”
35.180.234.186 - [26/Jan/2021:14:35:47 +0000] “GET /index.php” 404 0 - 24134 24467 0.303 27262976 56.19% 16.53% “/es/nada/nada”

This is me checking 404 file. The IPs are AWS Ezoic because my server is not recognizing some users IP

I found the following to avoid invalid traffic.
1)

BLOCK KNOWN HOSTING PROVIDERS AND PROXY SERVICES
Block these data centers:
DIGITAL OCEAN GIGENET OVH HOSTING CHOOPA, LLC

BLOCK End of Life More than 3 years
Firefox version < 52
Chrome version < 57
Internet Explorer version < 10
Safari version < 9

Is it okay to do this? If the answer is yes, how to block?

  1. You can block by ASN in Firewall Rules.
  2. As for older browser versions, that will be very tedious to block those by User Agent String. Your best bet is to enable Browser Integrity Check in Firewall → Settings.

I find that blocking by ASN and unwanted countries is quite effective.

Amazing. thanks!! this is what I was exactly needing.

I blocked in the IP Access Rules, its ok?

Down side, is there are not stats.

Hi I changed the ASN Blocking from IP Access Rules To WAF rules.

I have one WAF for allow good bots, and another WAF to block ASN. Is this WAF avoiding to block good bots when blocking ASN in another WAF?

I put “Allow” firewall rules first, then Challenge and Block rules later. If a bot matches the “Allow” rule, the other rules will not apply.

Thanks for your answer.
How many items can be added to the WAF?

WAF is a different feature, so this is for a Firewall Rule, as it’s not application-specific.

You’re allowed 5 active Firewall Rules on a free plan. Each rule an be as long as four thousand (probably 4096) characters.

1 Like

Hi @sdayman

Thanks blocking ASN was a great help. I can’t possibly believe what is going on with my site. According with Cloudflare analytics I have the following in last 24 hours:

  • Unique Visitors 5,080
  • Good Bots allow 9k
  • Bad bots block 262
  • ASN block 15,5k
    Yesterday was a half, the number was raised when I added AWS.
    Most from User agent axios/0.19.2 and fake Googlebot-Image/1.0 X-Middleton/1
  1. The bad bots block is working only partially, is not blocking bots with X-Middleton in the user agent, however ASN does.

  2. My server has not been able to restrict the traffic to only Cloudflare and Ezoic. I am afraid that I am blocking here and they will come directly without passing through Cloudflare. How can I check how much traffic is coming through Cloudflare and how much is coming directly?

  3. I think that I am still not getting the complete picture, because the request for last 24 hours were:

  • Argentina 46,240
  • United States 27,741
  • Colombia 5,237
  • Brazil 4,545
  • Peru 4,307

Your advise is very much appreciated

1 Like

Without this in place, it makes it difficult to track down the problem.

This is not easy. If you have the access and skill to track this down, then you have the access and skill to block traffic that doesn’t come through Cloudflare or Ezioc.

Where are you seeing this? And which traffic do you want?

Analytics> Top Traffic Countries / Regions

Last 24 hours

My site is for people that want to know about the country Argentina. 70% comes from Argentina, and 30% from Spanish speaking world. I have a summarized English translation, for targeting user interested in this country which is a small percentage.

I see that there is not solution for me if Cloudways don’t restrict the traffic

This topic was automatically closed after 30 days. New replies are no longer allowed.