Hello! We’ve been having a big problem with bots on our site lately, specifically Google crawlers. I run an online store selling art prints. We advertise on Facebook, Google and Pinterest. Every now and then we’ll get massive spikes in traffic from specific locations. This is usually because we launch new ads, and the bots from Facebook and Google will crawl our site to test the ads, so we see large spikes in traffic coming from cities where their data centers are located (see the city list below). While these spikes weren’t a problem before and would only come when we launched new ads, we noticed our bot traffic was getting worse and very
consistent. After filtering out the cities based on Facebook crawlers and Google crawlers, we noticed that it was the Google crawlers that were the issue (see the attached graph). Has anyone else seen a large spike in Google crawler traffic? This is a pain in the ass as I think it is having an effect on both our pixels in the store, our real traffic numbers, and is obviously skewing data in our analytics.
Facebook Bots:
- Luleå, Sweden
- Forest City, North Carolina
- Clonee, Ireland
- Altoona, Iowa
- Prineville, Oregon
Google Bots:
- Tukwila, Washington
- Fremont, California
- Cedar Knolls, New Jersey
- Miami, Florida (need to be specific to paid.Google)
- LA, California (need to be specific to paid.Google)
- Council Bluffs, Iowa
Amazon Centers:
(Can Block These Completely With No Threat to Ads)
- Ashburn, Virginia
Hetzner Data Centers:
(Can Block These Completely With No Threat to Ads)
- Saarbrücken, Germany
- Falkenstein, Germany
