Noindexing parameter/variant pages

siegfredalcover
Visitor
2 0 1

Hello.

 

I am new to shopify. I need to noindex variant pages of products, but I can't find anything after looking it up. I have products that have no variants but still have variant URLs which I need to somehow noindex and remove canonical tags from

An example page is this
Original Page - https://www.bareoutdoors.com.au/products/sorrento-6-seat-round-dining-setting

Variant Page - https://www.bareoutdoors.com.au/products/sorrento-6-seat-round-dining-setting?variant=40141629325365

It's literally the same product but it's both showing up in google causing duplicate content.

Replies 17 (17)

7tea_de
Tourist
3 0 1

Dear Support.

 

We have the same issue. Not only with variants.

The Googlebot and other bots crawls and index ugly pages (yes, we have canonical). But it should noindex.

https://7tea.de/products/bio-zitrone-brennnessel-minze-tee?pr_prod_strat=copurchase&pr_rec_id=c11e3b...

 

It's a waste of crawl budget, ugly URLs and duplicate content.

 

Normally, I would stop crawling with robots.txt for parameters.

Disallow: *?

 

But if indexed and crawled, it's only for the future. We can't remove it: It's blocked now (to prevent further requests).

I have seen here some hints, which I have tried with the theme.liquid.

Like:

 

{% if handle contains 'seq=uniform' %}
<meta name="robots" content="noindex">
{% endif %}

 

from here:

https://community.shopify.com/c/shopify-discussions/why-recommended-products-have-extra-params-in-ur...

 

But it seems, is has no influence inside the source code. It's index, follow furthermore.

Any idea? Many other people have the same issue.

 

@siegfredalcover 

I would add not nofollow. The crawler will break the crawling of other pages from here, and you have a lesser chance to remove the other bad URLs. Nofollow stop further crawling at this point. 

 

Noindex is fine.

Due noindex, it’s also a crawling budget invest for the future. It's reduced the attempts in the future
https://www.seroundtable.com/google-noindex-404-codes-crawl-budget-34510.html

 

Best

 

7

DestMan
Tourist
3 0 1

Hello,

I've fixed the issue above for my website using JavaScript

Use this code below:

<script>
document.addEventListener('DOMContentLoaded', function() {
if (window.location.href.indexOf('?') > -1) {
var metaNoIndex = document.createElement('meta');
metaNoIndex.name = 'robots';
metaNoIndex.content = 'noindex';
document.getElementsByTagName('head')[0].appendChild(metaNoIndex);
}
});
</script>

hope this helps!

Xavi8
Visitor
1 0 0

Hi Destman,

 

Where did you place this script? I've got the same issue, thousands of urls creatred from /search parameter, index, follow, so blocking is not an option until they are no index. Do you think it will work for me?

frankymartin
Shopify Partner
29 0 17

You must add the code in theme.liquid inside head tag.

frankymartin
Shopify Partner
29 0 17

Hi @DestMan and thanks!!

It works for me!!

sixthcitymkt1
Shopify Partner
5 0 0

@DestMan @frankymartin How did you test if this is working or not? I've implemented the code and have used a few different tools but I can't definitely tell if it's being indexed or not due to the structure of the javascript. 

 

Also - would you recommend throwing Disallow: *? into the robots.txt like @7tea_de suggests?

frankymartin
Shopify Partner
29 0 17

Hi @six 

 

You can know if that URLs are indexed or nor with Google Search Console or with "Robots Exclusion Checker" chrome extension (it also show the index status).

https://chromewebstore.google.com/detail/robots-exclusion-checker/lnadekhdikcpjfnlhnbingbkhkfkddkl

https://www.samgipson.com/robots-exclusion-checker-chrome-extension/

 

sixthcitymkt1
Shopify Partner
5 0 0

Amazing! This code

 

<script>
document.addEventListener('DOMContentLoaded', function() {
if (window.location.href.indexOf('?') > -1) {
var metaNoIndex = document.createElement('meta');
metaNoIndex.name = 'robots';
metaNoIndex.content = 'noindex';
document.getElementsByTagName('head')[0].appendChild(metaNoIndex);
}
});
</script>

 

 

worked for the "You May Also Like" pages such as this URL (to be no-indexed). Here's an example: 

 

https://www.paintedcowgirlwesternstore.com/products/copy-of-hooey-mens-canyon-black-hoodie-w-aztec-p...

 

However, I'm still seeing a variant page like this being indexed (using the chrome add on): 

 

https://www.paintedcowgirlwesternstore.com/products/copy-of-hooey-mens-canyon-black-hoodie-w-aztec-p...

 

Any additional help would be greatly appreciated. Thanks everyone! This is a huge win for us (even with the related products)

frankymartin
Shopify Partner
29 0 17

Hi @sixthcitymkt1 

I see a noindex in https://www.paintedcowgirlwesternstore.com/products/copy-of-hooey-mens-canyon-black-hoodie-w-aztec-p...

frankymartin_0-1703086982101.png

But you also need to block that URLs in robots with;

Disallow: *?

And you can ask to eliminate that URLs in Google Search Console.

sixthcitymkt1
Shopify Partner
5 0 0

Awesome - I think I have it but if you can take a quick look: https://www.paintedcowgirlwesternstore.com/robots.txt

 

I see it a few times in there 

frankymartin
Shopify Partner
29 0 17

Hi @sixthcitymkt1 

 

Your code is at tha bottom of the robots.txt, you must place the code in the first section User-agent: *

frankymartin_0-1703091979503.png

You can also add:

Disallow: /*?variant=*

 

sixthcitymkt1
Shopify Partner
5 0 0

Thanks! The code does seem to be working but I'm struggling to find where to place it in the robots.txt.liquid file. Here's the code in the file: 

 

# we use Shopify as our ecommerce platform
{%- comment -%}
# Caution! Please read https://help.shopify.com/en/manual/promoting-marketing/seo/editing-robots-txt before proceeding to make changes to this file.
{% endcomment %}


{% for group in robots.default_groups %}
{{- group.user_agent -}}

{% for rule in group.rules %}
{{- rule -}}
{% endfor %}

 

 

{%- if group.sitemap != blank -%}
{{ group.sitemap }}
{%- endif -%}
{% endfor %}


Disallow: *?

 

 

I'm seeing more URL's populating into blocked by robots.txt as search console has caught up with things. I'm still trying to figure out how to address the 36,000 404 errors in search console. Any thoughts on either would be greatly appreciated. 

DavideCart
Visitor
3 0 0

Hi, did you happen to solve anything?

Thanks 

gardentoolsbox
Visitor
1 0 1

Add the following codes at the end of your theme.liquid file. And then monitor your search console indexing results.

 

{%- if canonical_url="*?variant=*" -%}
<meta name="robots" content="noindex, nofollow">
{%- endif -%}

 

What's more, you can add Disallow: /*?variant=* on Robots testing tool, and test whether it can be indexed or not. Being blocked is the result you want. 

Willy

Tweetyx
Shopify Partner
1 0 0

For Seo, add in theme.liquid in the head that :

{%- capture contentForQuerystring -%}{{ content_for_header }}{%- endcapture -%}
{%- assign pageUrl = contentForQuerystring
| split: '"pageurl":"'
| last
| split: '"'
| first
| split: '.myshopify.com'
| last
| replace: '\/', '/'
| replace: '%20', ' '
| replace: '\u0026', '&'
-%}
{% capture finalurl %}https://{{ pageUrl }}{% endcapture %}

 


{% if finalurl contains "?" %}
{% if finalurl contains "?page" %}
{%- else -%}
<meta name="robots" content="noindex, nofollow">
{%- endif -%}
{%- endif -%}

DavideCart
Visitor
3 0 0

I have the same problem as what you mention in this article! Lots of problems of "Pages crawled, but not indexed" or "Alternative page with appropriate canonical tag", with all get variables, example this link: https://cartonlineitalia.it/products/sticco-stacco-crayola?pr_prod_strat=collection_fallback&pr_rec_...

I really don't know how to fix this, because this is penalizing us! at the crawl level in Wordpress and with Woocommerce everything was much better.

Does this code solve this problem? Where exactly should it be placed in the head tag?
URGENT thanks

thesafetysign
Visitor
1 0 0

Hello,

Did you manage to fix this? Im having the same issue. 

Im not as technical as most people. Im willing to pay someone to add these codes into my robot.txt file etc.

 

If anyone out there willing to do for me?