Googlebot Runs Through Cloudflare IP Addresses?


#21

I am sorry, yes in my logs. and yes I will post the IPS. These are all rapid firing consecutively. There are many more bot all start with these first 3 #s

108.162.245.136 Mozilla/5.0+(compatible;+bingbot/2.0;++http://www.bing.com/bingbot.htm)
162.158.106.30 Mozilla/5.0+(compatible;+bingbot/2.0;++http://www.bing.com/bingbot.htm)
108.162.246.47 Mozilla/5.0+(compatible;+bingbot/2.0;++http://www.bing.com/bingbot.htm)
108.162.237.148 Mozilla/5.0+(compatible;+Googlebot/2.1;++http://www.google.com/bot.html)
162.158.186.144 Mozilla/5.0+(compatible;+Googlebot/2.1;++http://www.google.com/bot.html)
162.158.106.132 Mozilla/5.0+(compatible;+bingbot/2.0;++http://www.bing.com/bingbot.htm)
108.162.237.124 Mozilla/5.0+(compatible;+Googlebot/2.1;++http://www.google.com/bot.html)
172.68.65.72 Mozilla/5.0+(compatible;+bingbot/2.0;++http://www.bing.com/bingbot.htm)
108.162.237.100 Mozilla/5.0+(compatible;+Googlebot/2.1;++http://www.google.com/bot.html)

This is the real Googlebot later on in the day

66.249.66.48 Mozilla/5.0+(compatible;+Googlebot/2.1;++http://www.google.com/bot.html) 307 0 0 78
66.249.66.1 Mozilla/5.0+(compatible;+Googlebot/2.1;++http://www.google.com/bot.html)


#22

These entries are from the same machine, the same log file, and there was not a single change made to the server configuration after the first batch and before the last batch of log entries?


#23

no. thats what is crazy. It seems to be scripts.


#24

What? My question was not referring to scripts at all? Can you please re-read my posting again and confirm or deny what I asked.


#25

I only copied snippets to show the IP and user agent

here is a small sample verbatum

2019-01-09 19:57:51 208.106.139.58 GET /NY/Huntington+Station-results.aspx - 443 - 66.249.66.44 Mozilla/5.0+(compatible;+Googlebot/2.1;++http://www.google.com/bot.html) 307 0 0 109
2019-01-09 19:57:56 208.106.139.58 GET /NY/Huntington+Station-results.aspx - 443 - 66.249.66.1 Mozilla/5.0+(compatible;+Googlebot/2.1;++http://www.google.com/bot.html) 200 0 0 3666
2019-01-09 20:00:12 208.106.139.58 GET /stockton-ports-tickets - 443 - 66.249.66.1 Mozilla/5.0+(Linux;+Android+6.0.1;+Nexus+5X+Build/MMB29P)+AppleWebKit/537.36+(KHTML,+like+Gecko)+Chrome/41.0.2272.96+Mobile+Safari/537.36+(compatible;+Googlebot/2.1;++http://www.google.com/bot.html) 200 0 0 885
2019-01-09 20:07:36 208.106.139.58 GET /tickets/house-of-blues-houston/t3830795/the-fab-5-the-beatles-tribute-band-and-diamond-jack-tickets - 443 - 40.77.167.178 Mozilla/5.0+(compatible;+bingbot/2.0;++http://www.bing.com/bingbot.htm) 200 0 0 780
2019-01-09 20:07:41 208.106.139.58 GET /harvest-at-jimmys-tickets - 443 - 207.46.13.35 Mozilla/5.0+(compatible;+bingbot/2.0;++http://www.bing.com/bingbot.htm) 200 0 0 936
2019-01-09 20:07:50 208.106.139.58 GET /tickets/camelback-ranch-stadium/t3688032/spring-training-los-angeles-dodgers-vs-los-angeles-angels-of-anaheim-ss-tickets - 443 - 66.249.66.1 Mozilla/5.0+(compatible;+Googlebot/2.1;++http://www.google.com/bot.html) 200 0 0 1123
2019-01-09 20:08:48 208.106.139.58 GET /venue/cox-activities-center-tickets-for-sale - 443 - 40.77.167.178 Mozilla/5.0+(compatible;+bingbot/2.0;++http://www.bing.com/bingbot.htm) 200 0 0 764
2019-01-09 20:09:26 208.106.139.58 GET / - 80 - 177.185.222.235 Mozilla/5.0+(Windows+NT+10.0;+WOW64)+AppleWebKit/537.36+(KHTML,+like+Gecko)+Chrome/52.0.2743.116+Safari/537.36 307 0 0 874
2019-01-09 20:09:30 208.106.139.58 GET /death-by-design-tickets - 443 - 157.55.39.75 Mozilla/5.0+(compatible;+bingbot/2.0;++http://www.bing.com/bingbot.htm) 200 0 0 733
2019-01-09 20:09:36 208.106.139.58 GET / - 443 - 18.188.240.21 Mozilla/5.0+(Macintosh;+Intel+Mac+OS+X+x.y;+rv:42.0)+Gecko/20100101+Chrome/67.0.3396.79+Safari/537.36 307 0 0 93
2019-01-09 20:09:39 208.106.139.58 GET /Default.aspx - 443 - 18.188.240.21 Mozilla/5.0+(Macintosh;+Intel+Mac+OS+X+x.y;+rv:42.0)+Gecko/20100101+Chrome/67.0.3396.79+Safari/537.36 200 0 0 343
2019-01-09 20:10:09 208.106.139.58 GET /shirley-caesar-tickets - 443 - 66.249.66.3 Mozilla/5.0+(Linux;+Android+6.0.1;+Nexus+5X+Build/MMB29P)+AppleWebKit/537.36+(KHTML,+like+Gecko)+Chrome/41.0.2272.96+Mobile+Safari/537.36+(compatible;+Googlebot/2.1;++http://www.google.com/bot.html) 200 0 0 702
2019-01-09 20:12:58 208.106.139.58 GET /alice-phoebe-lou-tickets - 443 - 40.77.167.178 Mozilla/5.0+(compatible;+bingbot/2.0;++http://www.bing.com/bingbot.htm) 200 0 0 1154
2019-01-09 20:14:24 208.106.139.58 GET /venue/paul-brown-stadium-parking-lots-tickets-for-sale - 443 - 207.46.13.33 Mozilla/5.0+(compatible;+bingbot/2.0;++http://www.bing.com/bingbot.htm) 200 0 0 1092
2019-01-09 20:14:33 208.106.139.58 GET /crown-college-storm-football-tickets - 443 - 40.77.167.178 Mozilla/5.0+(compatible;+bingbot/2.0;++http://www.bing.com/bingbot.htm) 200 0 0 1111
2019-01-09 20:16:48 208.106.139.58 GET /robots.txt - 80 - 54.36.149.70 Mozilla/5.0+(compatible;+AhrefsBot/6.1;++http://ahrefs.com/robot/) 307 0 0 171
2019-01-09 20:16:49 208.106.139.58 GET /robots.txt - 443 - 54.36.149.70 Mozilla/5.0+(compatible;+AhrefsBot/6.1;++http://ahrefs.com/robot/) 307 0 0 171
2019-01-09 20:16:50 208.106.139.58 GET /robots.txt - 443 - 54.36.149.70 Mozilla/5.0+(compatible;+AhrefsBot/6.1;++http://ahrefs.com/robot/) 200 0 0 171
2019-01-09 20:16:57 208.106.139.58 GET /colgate-red-raiders-soccer-soccer-tickets - 443 - 207.46.13.35 Mozilla/5.0+(compatible;+bingbot/2.0;++http://www.bing.com/bingbot.htm) 200 0 0 1162
2019-01-09 20:17:28 208.106.139.58 GET /tickets/vivint-smart-home-arena/t3741071/new-kids-on-the-block-salt-n-pepa-and-naughty-by-nature-tickets - 443 - 66.249.66.3 Mozilla/5.0+(compatible;+Googlebot/2.1;++http://www.google.com/bot.html) 200 0 0 1359
2019-01-09 20:17:40 208.106.139.58 GET /espn-college-gameday-hockey-tickets - 443 - 207.46.13.35 Mozilla/5.0+(compatible;+bingbot/2.0;++http://www.bing.com/bingbot.htm) 200 0 0 936
2019-01-09 20:17:47 208.106.139.58 GET /NY/Croton+On+Hudson-results.aspx - 443 - 66.249.66.44 Mozilla/5.0+(compatible;+Googlebot/2.1;++http://www.google.com/bot.html) 307 0 0 124
2019-01-09 20:17:53 208.106.139.58 GET /NY/Croton+On+Hudson-results.aspx - 443 - 66.249.66.1 Mozilla/5.0+(compatible;+Googlebot/2.1;++http://www.google.com/bot.html) 200 0 0 4807
2019-01-09 20:19:06 208.106.139.58 GET /venue/fantasy-springs-resort-and-casino-tickets-for-sale - 443 - 157.55.39.165 Mozilla/5.0+(compatible;+bingbot/2.0;++http://www.bing.com/bingbot.htm) 200 0 0 998
2019-01-09 20:19:37 208.106.139.58 GET /venue/tallinn-song-festival-ground-tickets-for-sale - 443 - 207.46.13.35 Mozilla/5.0+(compatible;+bingbot/2.0;++http://www.bing.com/bingbot.htm) 200 0 0 686
2019-01-09 20:20:08 208.106.139.58 GET /jordan-polovina-tickets - 443 - 66.249.66.3 Mozilla/5.0+(Linux;+Android+6.0.1;+Nexus+5X+Build/MMB29P)+AppleWebKit/537.36+(KHTML,+like+Gecko)+Chrome/41.0.2272.96+Mobile+Safari/537.36+(compatible;+Googlebot/2.1;++http://www.google.com/bot.html) 200 0 0 982
2019-01-09 20:21:25 208.106.139.58 GET /a-better-hand-tickets - 443 - 40.77.167.178 Mozilla/5.0+(compatible;+bingbot/2.0;++http://www.bing.com/bingbot.htm) 200 0 0 1357
2019-01-09 20:21:44 208.106.139.58 GET /venue/fuerza-bruta-tent-at-excalibur-hotel-and-casino-tickets-for-sale - 443 - 40.77.167.178 Mozilla/5.0+(compatible;+bingbot/2.0;++http://www.bing.com/bingbot.htm) 200 0 0 1248
2019-01-09 20:23:14 208.106.139.58 GET /bc-tickets - 443 - 173.245.54.7 Mozilla/5.0+(compatible;+bingbot/2.0;++http://www.bing.com/bingbot.htm) 307 0 0 93
2019-01-09 20:24:32 208.106.139.58 GET /bc-tickets - 443 - 40.77.167.178 Mozilla/5.0+(compatible;+bingbot/2.0;++http://www.bing.com/bingbot.htm) 200 0 0 717
2019-01-09 20:25:25 208.106.139.58 GET /elvis-the-exhibition-tickets - 443 - 207.46.13.33 Mozilla/5.0+(compatible;+bingbot/2.0;++http://www.bing.com/bingbot.htm) 200 0 0 577
2019-01-09 20:27:46 208.106.139.58 GET /tickets/the-wayfarer/t3785753/mozes-and-the-firstborn-and-the-parrots-tickets - 443 - 66.249.66.5 Mozilla/5.0+(compatible;+Googlebot/2.1;++http://www.google.com/bot.html) 200 0 0 842
2019-01-09 20:30:05 208.106.139.58 GET /buy-will-clarke-tickets - 443 - 66.249.66.1 Mozilla/5.0+(Linux;+Android+6.0.1;+Nexus+5X+Build/MMB29P)+AppleWebKit/537.36+(KHTML,+like+Gecko)+Chrome/41.0.2272.96+Mobile+Safari/537.36+(compatible;+Googlebot/2.1;++http://www.google.com/bot.html) 200 0 0 764
2019-01-09 20:32:59 208.106.139.58 GET /basketball-hall-of-fame-tip-off-gymnastics-tickets - 443 - 40.77.167.178 Mozilla/5.0+(compatible;+bingbot/2.0;++http://www.bing.com/bingbot.htm) 200 0 0 655
2019-01-09 20:34:37 208.106.139.58 GET /darci-lynne-tickets - 443 - 66.249.66.3 Mozilla/5.0+(compatible;+Googlebot/2.1;++http://www.google.com/bot.html) 200 0 0 748
2019-01-09 20:37:44 208.106.139.58 GET /IL/East+Saint+Louis-results.aspx - 443 - 66.249.66.46 Mozilla/5.0+(compatible;+Googlebot/2.1;++http://www.google.com/bot.html) 307 0 0 1185
2019-01-09 20:37:48 208.106.139.58 GET /IL/East+Saint+Louis-results.aspx - 443 - 66.249.66.3 Mozilla/5.0+(compatible;+Googlebot/2.1;++http://www.google.com/bot.html) 200 0 0 3469
2019-01-09 20:38:05 208.106.139.58 GET / - 443 - 18.224.184.7 Mozilla/5.0+(Windows+NT+6.1;+Win64;+x64;+rv:47.0)+Trident/5.0+Chrome/51.0.2704.106+Safari/537.36+OPR/38.0.2220.41 307 0 0 93
2019-01-09 20:38:06 208.106.139.58 GET /Default.aspx - 443 - 18.224.184.7 Mozilla/5.0+(Windows+NT+6.1;+Win64;+x64;+rv:47.0)+Trident/5.0+Chrome/51.0.2704.106+Safari/537.36+OPR/38.0.2220.41 200 0 0 639
2019-01-09 20:40:04 208.106.139.58 GET /lala-brass-band-tickets - 443 - 66.249.66.1 Mozilla/5.0+(Linux;+Android+6.0.1;+Nexus+5X+Build/MMB29P)+AppleWebKit/537.36+(KHT


#26

That still does not address my question and the most recent excerpt appears to be unrelated to what you posted earlier. Please address my earlier questions.


#27

Here is Google same day right at the end of the night. This is the real Google. starts with 66

2019-01-09 23:38:27 208.106.139.58 GET /elton-john-tickets.aspx - 443 - 66.249.66.3 Mozilla/5.0+(compatible;+Googlebot/2.1;++http://www.google.com/bot.html) 200 0 0 717
2019-01-09 23:38:59 208.106.139.58 GET / - 443 - 18.219.245.160 Mozilla/5.0+(X11;+Linux+x86_64)+AppleWebKit/537.36+(KHTML,+like+Gecko)+Chrome/67.0.3396.79+Safari/537.36 307 0 0 78
2019-01-09 23:39:01 208.106.139.58 GET /Default.aspx - 443 - 18.219.245.160 Mozilla/5.0+(X11;+Linux+x86_64)+AppleWebKit/537.36+(KHTML,+like+Gecko)+Chrome/67.0.3396.79+Safari/537.36 200 0 0 329
2019-01-09 23:39:27 208.106.139.58 GET /planet-axe-tour-tickets - 443 - 66.249.66.3 Mozilla/5.0+(Linux;+Android+6.0.1;+Nexus+5X+Build/MMB29P)+AppleWebKit/537.36+(KHTML,+like+Gecko)+Chrome/41.0.2272.96+Mobile+Safari/537.36+(compatible;+Googlebot/2.1;++http://www.google.com/bot.html) 200 0 0 814
2019-01-09 23:42:48 208.106.139.58 GET /scriptresource.axd d=qmh2ehwxps8qmeb3n6fheu0mxuv9_d4hlblybxdnq0wzbzdgbldcv3hcgdlfv362gknct3vzyrwfltve40tgjpr53guwno-cmiekzg4vgd00k-0kpmx0d11bxvic7-n2e2ze2hd4zajgcyeaj6rnlq2&t=ffffffffaa73f696 443 - 207.46.13.35 Mozilla/5.0+(compatible;+MSIE+9.0;+Windows+NT+6.1;+Trident/5.0) 307 0 0 78
2019-01-09 23:43:15 208.106.139.58 GET /scriptresource.axd d=qmh2ehwxps8qmeb3n6fheu0mxuv9_d4hlblybxdnq0wzbzdgbldcv3hcgdlfv362gknct3vzyrwfltve40tgjpr53guwno-cmiekzg4vgd00k-0kpmx0d11bxvic7-n2e2ze2hd4zajgcyeaj6rnlq2&t=ffffffffaa73f696 443 - 207.46.13.35 Mozilla/5.0+(compatible;+MSIE+9.0;+Windows+NT+6.1;+Trident/5.0) 404 0 0 46
2019-01-09 23:43:53 208.106.139.58 GET /robots.txt - 443 - 157.55.39.27 Mozilla/5.0+(compatible;+bingbot/2.0;++http://www.bing.com/bingbot.htm) 200 0 0 93
2019-01-09 23:44:27 208.106.139.58 GET /cycle-of-pain-tickets - 443 - 157.55.39.75 Mozilla/5.0+(compatible;+bingbot/2.0;++http://www.bing.com/bingbot.htm) 200 0 0 811
2019-01-09 23:44:31 208.106.139.58 GET /venue/the-jason-chin-harold-cabaret-at-io-chicago-tickets-for-sale - 443 - 157.55.39.75 Mozilla/5.0+(compatible;+bingbot/2.0;++http://www.bing.com/bingbot.htm) 200 0 0 624
2019-01-09 23:45:28 208.106.139.58 GET /venue/teatro-lauro-rossi-tickets-for-sale - 443 - 207.46.13.204 Mozilla/5.0+(compatible;+bingbot/2.0;++http://www.bing.com/bingbot.htm) 200 0 0 670
2019-01-09 23:45:30 208.106.139.58 GET /venue/bernard-b-jacobs-theater-tickets-for-sale - 443 - 207.46.13.204 Mozilla/5.0+(compatible;+bingbot/2.0;++http://www.bing.com/bingbot.htm) 200 0 0 718
2019-01-09 23:47:06 208.106.139.58 GET /venue/kool-haus-tickets-for-sale - 443 - 66.249.66.46 Mozilla/5.0+(compatible;+Googlebot/2.1;++http://www.google.com/bot.html) 307 0 0 171
2019-01-09 23:47:07 208.106.139.58 GET /venue/kool-haus-tickets-for-sale - 443 - 66.249.66.5 Mozilla/5.0+(compatible;+Googlebot/2.1;++http://www.google.com/bot.html) 200 0 0 577
2019-01-09 23:47:46 208.106.139.58 GET /enrique-batiz-tickets - 443 - 207.46.13.35 Mozilla/5.0+(compatible;+bingbot/2.0;++http://www.bing.com/bingbot.htm) 200 0 0 624
2019-01-09 23:48:18 208.106.139.58 GET /abba-magic-tickets - 443 - 66.249.66.3 Mozilla/5.0+(compatible;+Googlebot/2.1;++http://www.google.com/bot.html) 200 0 0 561
2019-01-09 23:49:16 208.106.139.58 GET /gimme-abbey-the-beatles-and-rolling-stones-tribute-band-tickets - 443 - 207.46.13.204 Mozilla/5.0+(compatible;+bingbot/2.0;++http://www.bing.com/bingbot.htm) 200 0 0 546
2019-01-09 23:49:25 208.106.139.58 GET /le-butcherettes-tickets - 443 - 66.249.66.5 Mozilla/5.0+(Linux;+Android+6.0.1;+Nexus+5X+Build/MMB29P)+AppleWebKit/537.36+(KHTML,+like+Gecko)+Chrome/41.0.2272.96+Mobile+Safari/537.36+(compatible;+Googlebot/2.1;++http://www.google.com/bot.html) 200 0 0 780
2019-01-09 23:57:04 208.106.139.58 GET /tickets/marvel-stadium/t3729229/taylor-swift-tickets - 443 - 66.249.66.3 Mozilla/5.0+(compatible;+Googlebot/2.1;++http://www.google.com/bot.html) 200 0 0 561
2019-01-09 23:58:20 208.106.139.58 GET /tickets/scotiabank-saddledome/t3734833/twenty-one-pilots-tickets - 443 - 66.249.66.5 Mozilla/5.0+(compatible;+Googlebot/2.1;++http://www.google.com/bot.html) 200 0 0 951
2019-01-09 23:59:23 208.106.139.58 GET /shoreline-mafia-tickets - 443 - 66.249.66.5 Mozilla/5.0+(Linux;+Android+6.0.1;+Nexus+5X+Build/MMB29P)+AppleWebKit/537.36+(KHTML,+like+Gecko)+Chrome/41.0.2272.96+Mobile+Safari/537.36+(compatible;+Googlebot/2.1;++http://www.google.com/bot.html) 200 0 0 924


#28

These are server log files yes. And I am waiting for the server files from my other site to be sent to me. I have 5 months worth of server files on Ticketron.com. I believe I answered all of the above questions. I will take this to Cloudflare support next.


#29

Okay.


#30

As you can see there is an issue. I showed you a very small sample of the IPs from Cloudflare. There are many more. And I have many other proofs. I am working with decoders now and awaiting their findings. We are trying to find where this traces back to directly so we can prosecute


#31

Already have the builder of the program and he will talk.


#32

You said you are going to take this to support, so there is no point in continuing here.


#33

Keep in mind that @ points to the server, www goes through Cloudflare, therefore it is entirely likely that traffic from similar bots intermittently does and does not go through Cloudflare.


#34

Thats a valid point, but without knowing what is configured and without too much cooperation on the OP’s side it all is relatively pointless anyhow.


#35

both www and @ are set to run through cloudflare servers.


#36

I have cooperated as much as I can. Everything you have asked I answered


#37

They are not

Asked twice, both times you answered only partially


#38

its not intermittent though its rapid fire all day long. both the @ and www are orange going through cloudflare


#39

Again, they are not. And @thedaveCA’s explanation is probably the most realistic one. You are not rewriting IP addresses, hence Cloudflare’s IPs show up for Cloudflare connections and all the proper addresses come from the direct connections. But that could have been clarified three hours ago already.


#40

is this what you are talking about?

Automatic HTTPS Rewrites

Automatic HTTPS Rewrites helps fix mixed content by changing “http” to “https” for all resources or links on your web site that can be served with HTTPS.

This setting was last changed 4 months ago