FROM CACHE - en_header

What's the recommended way of blocking bot spam (from a single source)?

Matt_T2
Shopify Partner
6 0 2

Hello,

Our website is getting hammered with bot spam. It all originates from Boardman, Oregon with the hostname "amazon" -- so it's clearly from Amazon's data center at that location. Thankfully, the bots aren't signing up or checking out, but their direct visits now comprise 50+% (!!!) of our daily traffic. This volume has destroyed the usefulness of our Shopify dashboard/metrics... thankfully, we can filter it out in Google Analytics.

What we're looking for is a way to block this bot spam. What is Shopify's official, recommended means of blocking bot spam, since we don't have access to server files (such as .htaccess) to do so ourselves? Are apps like TrafficGuard or Visitor Blocker the recommended method? We're hoping for a comprehensive answer that can help the community as well as ourselves.

I've seen a few threads asking similar questions, and haven't been impressed by the responses. Please note that we're not looking for advice on how to filter out this traffic in GA, nor are we gathering others' opinions on the harmfulness of bot spam on SEO or SERP rankings. "Just don't worry about it" is not a fix. We're looking for a real solution; if a teenager can fix it in five minutes with .htaccess, then a $16B-market-cap company and its intelligent community can also find a solution.

Thanks!

Replies 46 (46)
ArtinCoins
Tourist
3 0 4

I'm afraid we are in the same boat here at Art in Coins and I'm rather disappointed that Shopify doesn't have any server side tools in place to assist.  That's what I would expect with what I pay to use this platform and I'm not at all pleased to find that my only recourse is a very expensive third party app.  (such as Traffic Guard / Visitor Blocker) 

C'mon Shopify, time to get on the ball (please) and get some tools in place to help users block unwanted traffic.

Betty2
New Member
2 0 7

Hmmm . . . no offical response.

From what I'm seeing elsewhere this started being a problem for a wider audience (not just shopify) as of June 19th.

Disappointing.

Robin_Burton
Tourist
10 0 6
ArtinCoins
Tourist
3 0 4

During my investigation, I have come across the following bash script on Github that can be employed as a server side solution.  I sent the information to Shopify support to investigate and hopefully deploy for their users.  The script is for either Unix or Linux servers so hopefully compatible with Shopify.

AWS-BLOCKER
A simple bash script to block all AWS IP ranges using iptables.
https://github.com/corbanworks/aws-blocker/blob/master/aws-blocker

#!/bin/bash -e
#
# Amazon AWS blocker through iptables.
#
# First we use curl to grab the official list of ranges from Amazon. The -s
# prevents extraneous output from curl, and the -L makes it follow redirects.
#
# The ranges are passed to jq, a JSON parser. The -r makes jq output raw data
# without quotes. We only need the list of prefixes, so we discard everything
# else.

POSITION=1
FILTERS=""
JSON_URL="https://ip-ranges.amazonaws.com/ip-ranges.json"


# Get the line where the jump will be inserted at.
# Useful if you want e.g related / established rules for outgoing traffic.
if [[ -n $1 ]]; then
    POSITION=$1
    shift
fi


##
# Builds region filters based on CLI arguments
#
# Arguments: CLI arguments as passed by $*
#
function build_filters() {
    for arg in ${@:1}; do
        if [[ -n $filters ]]; then
            filters=$filters", "
        fi

        filters=$filters"select(.region | contains(\"$arg\"))"
    done

    if [[ -n $filters ]]; then
        filters=" | "$filters
    fi

    echo $filters
}


##
# Extracts IP ranges from an Amazon JSON file
#
# Arguments:
#     $1 AWS JSON content
#     $2 Prepared filter string
#     $3 Group to extract IP ranges from (e.g. prefixes)
#     $4 Object key for IP ranges (e.g ip_prefix)
#
function extract_ip_ranges() {
    local json=$1
    local filters=$2
    local array=$3
    local prefix=$4

    local group='group_by(.'$prefix')'
    local map='map({ "ip": .[0].'$prefix', "regions": map(.region) | unique, "services": map(.service) | unique })'

    local to_string='.ip + " \"" + (.regions | sort | join (", ")) + "\" \"" + (.services | sort | join (", ")) + "\""'
    local process='[ .'$array"[]$filters ] | $group | $map | .[] | $to_string"

    local ranges=$(echo "$json" | jq -r "$process" | sort -Vu)
    echo "$ranges"
}


##
# Creates the AWS iptables chain if it doesn't exist, then flushes it
#
# Arguments:
#     $1 Version to use. Omit for v4
#     $2 Position to insert chain statement at
#
function create_and_flush_chain() {
    local version=$1
    local position=$2
    local cmd=ip${version}tables

    $cmd -n --list AWS >/dev/null 2>&1 \
        || ($cmd -N AWS && $cmd -I INPUT $position -j AWS)

    $cmd -F AWS
}


##
# Adds an iptables rule for each line in ranges
#
# Arguments:
#     $1 Version to use. Omit for v4
#     $2 Prepared lines
#
function add_iptables_rules() {
    local version=$1
    local cmd=ip${version}tables
    local lines
    local data

    IFS=$'\n' lines=($2)
    unset IFS

    for line in "${lines[@]}"; do
        eval local data=($line)
        local ip=${data[0]}
        local regions=$(echo ${data[1]} | tr '[:upper:]' '[:lower:]')
        local services=$(echo ${data[2]} | tr '[:upper:]' '[:lower:]')

        $cmd -A AWS -s "$ip" -j REJECT -m comment --comment "$regions = $services"
    done
}

# Retrieve IP ranges definition
# Either from an URL or file input (e.g. "< ranges.json")
if [ ! -t 0 ]; then
    JSON=$(cat - <&0)
else
    JSON=$(curl -s -L $JSON_URL)
fi

FILTERS=$(build_filters "$*")


# IPv4
create_and_flush_chain "" $position
V4_RANGES=$(extract_ip_ranges "$JSON" "$FILTERS" "prefixes" "ip_prefix")
add_iptables_rules ""  "$V4_RANGES"


# IPv6
create_and_flush_chain 6 $position
V6_RANGES=$(extract_ip_ranges "$JSON" "$FILTERS" "ipv6_prefixes" "ipv6_prefix")
add_iptables_rules "6" "$V6_RANGES"

James96
New Member
2 0 1

Has anyone made any progress on this?

German_Toy_Stor
New Member
4 0 0

Same problem here! Any progress on that topic?

James96
New Member
2 0 1

Doesnt appear so. Its creating malformed URL's on my so I now have a huge amount of 404's.

German_Toy_Stor
New Member
4 0 0

We actually have two shops which are affected. Several emails with Shopify, but so far they only referred to 3rd party apps. Absolutely not satisfying. 

ArtinCoins
Tourist
3 0 4

Hello all,

I have been in recent email discussions with one of Shopify's network specialists and they are looking at some options to combat this issue.  One thing I am working on is compiling a list of IP blocks that this Boardman Bot is using to spam my wesbite.

What is see used most is ip address starting with:

18, 32, 34 & 54

If anyone has observed other IP block's please post them here and I'll be happy to communicate them with Shopify.  I'm happy they are taking note and looking to take action either on a case by case basis, or, if this is now widespread, a broad platform solution so we won't have to resort to expensive 3rd party apps.

Speaking of apps.  I use the following:  Back in Stock, Delerious Profit, Mailchimp, Order Printer, Product Reviews and Tawk.to Chat.  I'm taking a little poll to see if perhaps there's an app on the platform that's got a malicious backdoor to it that's directing this bot activity.  Not likely since the Boardman Bot is bombing tons of websites and not just Shopify's, but I am very curious about how this fake search traffic got turned on to my particular store.

Cheers all, light at the end of the tunnel.

Olive_McKillop1
New Member
1 0 5

Well it looks like loads of Shopify clients have been trying to get an answer to the problem of stopping spam bots from subscribing to their website with no response from Shopify.  I too have this irritation.  Shopify when are we going to get an answer from you that works.  Dont see why we should have to pay other organisations for an App that you can produce!  Angry here in Australia!

Katlin
New Member
1 0 2

Same issues here. 3/4 of my traffic is now some Amazon bot, and apparently there is no way to stop it. Thanks Shopify.

moonspinners
New Member
2 0 1

I have been having the same issue as well from about the middle of June.  Somebody needs to find an inexpensive, and preferably free, fix to this problem. Shopify help us out please!!

Spirit_Wanderer
Tourist
8 0 2

That is awesome. My traffic over time was replaced by 98% bots.

gm412
New Member
1 0 1

Same problem here from Boardman Oregon and Kansas. This should be an easy fix on server side but so far seems like Shopify is unwilling to do anything.

Spirit_Wanderer
Tourist
8 0 2

 

I lost two online stores on Dec 26, 2019 due to BOT attacks and DDoS attacks. Since then, I have tried to make the shopify experts see the harm these things are doing to all of us. The replies I have been getting from the technical department staff are stupid ones like, it is your third party app's, these are Google bots (when in fact they are FAKE GOOGLE BOTS on the entirely wrong Ip address) or my favorite in the case of Oberlo imports that do not bring all the information of the product in the import or need 5 refreshes to get the pics in, apparently we are making mistakes when selecting data ..... and blah blah blah!

 

Today I received another answer that nothing is dangerous for my business, yet Cloudflare (yes we moved our stores out of Shopify's reach to cloudflare) tells me a very different story. WHAT THEB F... K, Like i'm supposed to believe in their "infaillble protection) ! NO WAY!!!!

 

So we opened another shop, kept it under wraps, did EVERYTHING to hide it from the bots and as soon as it was online the bots started up and then kept on coming.

 

So now i have 3 online stores


When nobody knows you are opening a  online Store, that you put your password protect every page, and supposedly not one human can enter your store,  Now please explain to me how you wind up with 500 visits to your site and your password gets changed for Botwaw on the page that protects your store that bots aren't harmful.

 

You should be able to ask questions to the "Experts" and get an intelligent answer.


I know I'm not the only one who has "bot" problems and I have proof and data that the server where is my store is located receives  AUTOMATIC bot attacks and that these attacks are called on by the Shopify server, I'm talking about the server that houses ALL of our collective stores under ONE SINGLE IP ADDRESS. Did. you know that this server is actually red flagged in IPWatson.com as containing at least 2 major malwares AND IS FLAGGED AS A VERY HIGH RISK OF HACKING OR BEING HACKED?

 

I ask all those who have the same problems and who are tired of trying to make Shopify understand the situation and who make are thought of themselves as being crazy. That's what happens to me and the answers I get imply that I don't know anything.

 

Can you also tell me how these bot attack the stores of Shopify using RUBY, LIQUID PYTHON NUMPY, PYTHON BOOLEEN.

 

When I ask, I get this answer "It is impossible our protection is impenetrable".


WTF !!!!!!! We have proven it isn't and nobody is listening.

 

 

Those who are really tired of this fight with SHOPIFY, email me at spiritwanderer2018@gmail.com or fashionlimonde@gmail.com

 

Because the police have my files and I work with them to supply them all the evidence I have.

Because according to the law I HAVE THE RIGHT TO SUE SHOPIFY BECAUSE THEY AREN'T  DOING ANYTHING AND AS FAR AS WE ARE CONCERNED THEY ARE IN BREACH OF CONTRACT. WE CAN ALSO ACCUSE THEM OF BEING RESPONSIBLE FOR THESE ATTACKS.

bonhommie
New Member
1 0 1

We've been on Shopify Advanced for almost 5 years now and never noticed any bot issues but since Feb 9, 2020, we've had hundreds of daily spy bot visits! I thought this would be an easy/quick fix and I'm shocked to see that Shopify has no way to combat/block malicious spam bots. Definitely following this thread.

FlairTrade
New Member
2 0 1

Just experienced this for the first time from Boardman, OR and Kansas. Can't believe they haven't implemented some authentication process for legit traffic. It's possible many of the APP providers use AWS so some of this maybe legit traffic but it does pollute the analytics data.

Greenbobbin
Excursionist
23 0 5

So we have a new website and new to shopify for about 4 weeks (great timing I know!) but over half our traffic is coming in from Boardman, Oregon. Clearly its not real traffic and we dont even sell in the US. So what are folks doing about it? Has Shopify done anything to help?

Zoolander
Excursionist
11 0 8

I asked Shopify about this two weeks ago and even requested the IP address which they could or would not provide. We are all spending a lot of time and / or money trying to resolve what is probably a very simple fundamental issue for Shopify to resolve or block.  If I find a solution I will post it on the Shopify For Beginners Facebook page