We've discovered that Google is indexing our search result pages that include all sorts of dodgy pharma terms and URLs, e.g. /search/?q=Order%20Piroxicam%20🛏%EF%B8%8F🐇🥘%20Feldene%20Piroxicam%2020%20Mg%20Tablet%20-%20💠%20www.DoctorFox.store%20💠%20Pills%20without%20prescription%20🥘🐇🛏%EF%B8%8F%2010%20Mg%20Price%2C%20Piroxicam%20Gel%20Price%20Uk . The issue is getting bigger and worse every week. This is far from the website offering (hats). Of course, the search results for these 'terms' are blank/no products shown... but the page loads.
We've disavowed the linking domains in Google Search Console but this is just a patch and we need an actual solution.
While Shopify's default robots.txt disallows access to '/search' URLs, they still get indexed in Google which can potentially lead to Google penalty. Normally, I would do what Google suggests and just remove the 'Disallow: /search' rule from the robots.txt and then add a 'noindex,nofollow' meta robots tag to all '/search' URLs. BUT, of course, Shopify doesn't allow changing the robots.txt file so I can't remove the Disallow rule... which means Googlebot will never actually get to see that 'noindex,nofollow' meta robots tag!
Another route we tried was to 301 redirect all matching dodgy search URLs to a 404 page with the hope that this will stop whoever/whatever is creating these links from continuing with this tactic. BUT, again, Shopify doesn't allow 301 redirecting live/200 URLs. So we're sort of stuck.
Maybe this could be resolved with a better Search app? If there was a search app that did not allow loading a results page with 0 products?
Just a quick update about the robot.txt file that was referenced in this post. As of today, June 21st, 2021, we have launched the ability to edit the robot.txt file to give merchants more control over the information that is crawled by search engines. You can learn more about how to edit your robot.txt file through our community post here.
If you have any questions about the new feature, please do not hesitate to create a new post under our "Techincal QA" board.
Would you love to unleash the unbridled power of the Google Shopping Channel into your sho...By Gabe Jan 6, 2023
How can you turn a hobby into a career? That’s what Emmanuel did while working as a wa...By Skye Dec 30, 2022