FROM CACHE - en_header

Stopping unwanted on-site pharma search results

vader_art
Tourist
3 0 0

Hi

We've discovered that Google is indexing our search result pages that include all sorts of dodgy pharma terms and URLs, e.g. /search/?q=Order%20Piroxicam%20🛏%EF%B8%8F🐇🥘%20Feldene%20Piroxicam%2020%20Mg%20Tablet%20-%20💠%20www.DoctorFox.store%20💠%20Pills%20without%20prescription%20🥘🐇🛏%EF%B8%8F%2010%20Mg%20Price%2C%20Piroxicam%20Gel%20Price%20Uk . The issue is getting bigger and worse every week. This is far from the website offering (hats). Of course, the search results for these 'terms' are blank/no products shown... but the page loads.

We've disavowed the linking domains in Google Search Console but this is just a patch and we need an actual solution.

While Shopify's default robots.txt disallows access to '/search' URLs, they still get indexed in Google which can potentially lead to Google penalty. Normally, I would do what Google suggests and just remove the 'Disallow: /search' rule from the robots.txt and then add a 'noindex,nofollow' meta robots tag to all '/search' URLs. BUT, of course, Shopify doesn't allow changing the robots.txt file so I can't remove the Disallow rule... which means Googlebot will never actually get to see that 'noindex,nofollow' meta robots tag!

Another route we tried was to 301 redirect all matching dodgy search URLs to a 404 page with the hope that this will stop whoever/whatever is creating these links from continuing with this tactic. BUT, again, Shopify doesn't allow 301 redirecting live/200 URLs. So we're sort of stuck.

Maybe this could be resolved with a better Search app? If there was a search app that did not allow loading a results page with 0 products?

Reply 1 (1)
Trevor
Community Moderator
Community Moderator
3266 439 819

Hello!

Just a quick update about the robot.txt file that was referenced in this post. As of today, June 21st, 2021, we have launched the ability to edit the robot.txt file to give merchants more control over the information that is crawled by search engines. You can learn more about how to edit your robot.txt file through our community post here

If you have any questions about the new feature, please do not hesitate to create a new post under our "Techincal QA" board.

Trevor | Community Moderator @ Shopify
 - Was my reply helpful? Click Like to let me know! 
 - Was your question answered? Mark it as an Accepted Solution
 - To learn more visit the Shopify Help Center or the Shopify Blog