Blocking a Page from being Indexed with Theme.Liquid not working

PossumProgram
New Member
3 0 2

So as it's known, we cannot edit our Robots.txt to block a URL from being indexed by Google. I've followed the instructions on the documentation and added this to my header of the Theme.Liquid

{% if page.handle == 'dont-index-me' %}
      <meta name="robots" content="noindex,nofollow">
{% endif %}

When I go to the search console and perform a live URL test, Google has zero issues seeing the page. I also don't see the page or handle on the Robots.txt file when I look at it through the MyStoreURLaddress/Robots.txt.

What am I doing wrong? How long does it take for the page to be added to my Robots.txt file and so forth?  I have quadruple checked the handle, waited some time and gone to the Search Console and done live URL checks rather than checking the history. Running out of ideas. 

0 Likes

Hi!

In my experience, it usually takes between 24 to 48 hours for the robot.txt to be updated in Shopify.

Furthermore, make sure you're using the exact syntax suggested by Shopify to avoid any possible issues.


◦ Product page slider & thumbnails with swiping for free themes: Minimal | Debut | Brooklyn | Supply
◦ Replace apps with simple copy/paste code snippets. Examples clicking here.
0 Likes
Jason
Shopify Expert
10360 158 2009

What am I doing wrong? How long does it take for the page to be added to my Robots.txt file and so forth? 

Adding the robots meta tag has does not impact the robots.txt file in any way. Those are different things.

 

 I've followed the instructions on the documentation and added this to my header of the Theme.Liquid

Now the code you added doesn't account for the fact that you (likely) also have another robots meta tag in place. Have you made sure that when your noindex meta tag is shown that you are also removing the other?

 

I jump on these forums to help and share some insights. Not looking to be hired, and not looking for work.

Don't hand out staff invites or give admin password to forum members unless absolutely needed. In most cases the help you need can be handled without that.


★ http://freakdesign.com.au ★
0 Likes
Trevor
Community Moderator
Community Moderator
2570 337 524

Hello!

Just a quick update about the robot.txt file that was referenced in this post. As of today, June 21st, 2021, we have launched the ability to edit the robot.txt file to give merchants more control over the information that is crawled by search engines. You can learn more about how to edit your robot.txt file through our community post here

If you have any questions about the new feature, please do not hesitate to create a new post under our "Techincal QA" board.

Trevor | Community Moderator @ Shopify 
 - Was my reply helpful? Click Like to let me know! 
 - Was your question answered? Mark it as an Accepted Solution
 - To learn more visit the Shopify Help Center or the Shopify Blog

0 Likes