robots.txt not working

Rocking
Explorer
45 1 12

I am adding the correct format into the theme.liquid file but a week later and I still don't see these page in the my-store-domain.com/robots.txt file.

 

{% if handle contains 'test' %}
<meta name="robots" content="noindex">
{% endif %}

{% if handle contains 'tmp' %}
<meta name="robots" content="noindex">
{% endif %}

What am I doing wrong?

0 Likes
Jason
Shopify Expert
10360 158 2007

Adding code into your Liquid template has no impact on what is contained within the robots.txt file.
What are you expecting to see in that file?

I jump on these forums to help and share some insights. Not looking to be hired, and not looking for work.

Don't hand out staff invites or give admin password to forum members unless absolutely needed. In most cases the help you need can be handled without that.


★ http://freakdesign.com.au ★
0 Likes
Rocking
Explorer
45 1 12

All I want is to hide\exclude certain pages from SEO searches. I understand that we don't have direct access to the robots.txt file and adding that code to the theme.liquid was what I read to do on many forums. How do you suggest to exclude a page from Google's search?

0 Likes
Rocking
Explorer
45 1 12

Shopify says in their help pages on https://help.shopify.com/en/manual/promoting-marketing/seo/hide-a-page-from-search-engines 

 

You can hide pages that are not included in your robots.txt file by customizing the <head> section of your store's theme.liquid layout file. You need to include some code to noindex particular pages

 

This is what doesn't seem to work for me.

0 Likes
Jason
Shopify Expert
10360 158 2007

So this isn't about robots.txt and more about adding a noindex tag. Let's revisit your earlier comment now that we've cleared that up.

 

Looking at this code...

 

{% if handle contains 'test' %}
<meta name="robots" content="noindex">
{% endif %}

{% if handle contains 'tmp' %}
<meta name="robots" content="noindex">
{% endif %}

... I can see that if the handle has either "test" or  "tmp" somewhere in it, that metatag will be shown. If you're saying it's not working can you link to such a page on your site?

 

Are you also sure that your theme doesn't have an existing meta robots tag? You don't want to have two of them. See a link to one of the pages will very quickly determine what's happening.

 

I jump on these forums to help and share some insights. Not looking to be hired, and not looking for work.

Don't hand out staff invites or give admin password to forum members unless absolutely needed. In most cases the help you need can be handled without that.


★ http://freakdesign.com.au ★
0 Likes
HansMast
New Member
2 0 0

You're confusing two different ways of giving robot directives to Google:

  • robots.txt for the whole domain
  • Meta Robots tags on each individual page

They accomplish some of the same things, but will show up in very different places.

The code you're using adds a Meta Robots tag to an individual page. You can see that code by Viewing Page Source while on the page.

Adding a Meta Robots tag will not update robots.txt

0 Likes
Trevor
Community Moderator
Community Moderator
2569 336 523

Hello!

As of today, June 21st, 2021, we have launched the ability to edit the robot.txt file to give merchants more control over the information that is crawled by search engines. You can learn more about how to edit your robot.txt file through our community post here

Due to the age of the topic, I will be locking this thread. If you have any questions about the new feature, please do not hesitate to create a new post under our "Techincal QA" board.

Trevor | Community Moderator @ Shopify 
 - Was my reply helpful? Click Like to let me know! 
 - Was your question answered? Mark it as an Accepted Solution
 - To learn more visit the Shopify Help Center or the Shopify Blog