robots.txt not working

Highlighted
Explorer
45 1 10

I am adding the correct format into the theme.liquid file but a week later and I still don't see these page in the my-store-domain.com/robots.txt file.

 

{% if handle contains 'test' %}
<meta name="robots" content="noindex">
{% endif %}

{% if handle contains 'tmp' %}
<meta name="robots" content="noindex">
{% endif %}

What am I doing wrong?

0 Likes
Highlighted
Shopify Expert
10007 116 1817

Adding code into your Liquid template has no impact on what is contained within the robots.txt file.
What are you expecting to see in that file?

★ Winning Partner of the Build a Business competition. ★ http://freakdesign.com.au
0 Likes
Highlighted
Explorer
45 1 10

All I want is to hide\exclude certain pages from SEO searches. I understand that we don't have direct access to the robots.txt file and adding that code to the theme.liquid was what I read to do on many forums. How do you suggest to exclude a page from Google's search?

0 Likes
Highlighted
Explorer
45 1 10

Shopify says in their help pages on https://help.shopify.com/en/manual/promoting-marketing/seo/hide-a-page-from-search-engines 

 

You can hide pages that are not included in your robots.txt file by customizing the <head> section of your store's theme.liquid layout file. You need to include some code to noindex particular pages

 

This is what doesn't seem to work for me.

0 Likes
Highlighted
Shopify Expert
10007 116 1817

So this isn't about robots.txt and more about adding a noindex tag. Let's revisit your earlier comment now that we've cleared that up.

 

Looking at this code...

 

{% if handle contains 'test' %}
<meta name="robots" content="noindex">
{% endif %}

{% if handle contains 'tmp' %}
<meta name="robots" content="noindex">
{% endif %}

... I can see that if the handle has either "test" or  "tmp" somewhere in it, that metatag will be shown. If you're saying it's not working can you link to such a page on your site?

 

Are you also sure that your theme doesn't have an existing meta robots tag? You don't want to have two of them. See a link to one of the pages will very quickly determine what's happening.

 

★ Winning Partner of the Build a Business competition. ★ http://freakdesign.com.au
0 Likes
Highlighted
New Member
2 0 0

You're confusing two different ways of giving robot directives to Google:

  • robots.txt for the whole domain
  • Meta Robots tags on each individual page

They accomplish some of the same things, but will show up in very different places.

The code you're using adds a Meta Robots tag to an individual page. You can see that code by Viewing Page Source while on the page.

Adding a Meta Robots tag will not update robots.txt

0 Likes