Exclude a page from google

Solved
Brian67
Excursionist
61 0 2


ok so having poked around the forums a bit, I've learned now that I cannot edit the robots.txt file so I'm looking to see if anyone has another solution to "hide" a page from Google and other search engines?

As part of a promotion with another site, I've had to create a special page on my site where the others sites "premium members" can get exclusive coupon codes for my site.     Now these are pretty good coupon codes (that don't expire) so I'd rather not have them indexed and searchable.

Thoughts on how to hide this page?

Accepted Solution (1)

Accepted Solutions
Trevor
Community Moderator
Community Moderator
3214 435 743

This is an accepted solution.

Hello!

As of today, June 21st, 2021, we have launched the ability to edit the robot.txt file to give merchants more control over the information that is crawled by search engines. You can learn more about how to edit your robot.txt file through our community post here

Due to the age of the topic, I will be locking this thread and marking it as resolved to help direct anyone that lands on this page. If you have any questions about the new feature, please do not hesitate to create a new post under our "Techincal QA" board.

Trevor | Community Moderator @ Shopify 
 - Was my reply helpful? Click Like to let me know! 
 - Was your question answered? Mark it as an Accepted Solution
 - To learn more visit the Shopify Help Center or the Shopify Blog

View solution in original post

Replies 12 (12)
Samuel_Caleb
Shopify Expert
152 0 13

You could probably do this with Google Webmaster Tools, I think? You have to register your site first, and embed a piece of code into the head, then submit a removal request under crawler access. Apparently with Robots.txt it might still pop up again though. Otherwise you can use a noindex metatag I think. I'm not terribly sure but I hope this helps. Good luck!

Blogger by day at ReferralCandy, crime-fighter by night as Bill Nye the Science Guy.
Jason
Shopify Expert
10882 190 2179

Otherwise you can use a noindex metatag I think

Yep - that's it! Add meta tags to the header of those pages you don't want indexed. Of course, if the search engine wants to just ignore these, they can. Google should honour them.

<meta name="robots" content="noindex">

To allow other robots to index the page on your site, preventing only Google's robots from indexing the page:

<meta name="googlebot" content="noindex">

See this page for more info.

 

Apparently with Robots.txt it might still pop up again though

and it will still appear in the site map too...

★ I jump on these forums in my free time to help and share some insights. Not looking to be hired, and not looking for work. http://freakdesign.com.au ★
Helen_Linley
Tourist
12 0 2

Is there a way to exclude a page/product from the sitemap.xml. Google frowns upon you including non-indexed pages in your sitemap.

Thanks

Helen_Linley
Tourist
12 0 2

Just realised you can tell google to use a different sitemmap url so I could just create my own. 

Would it be possible for Shopify to publish the liquid template it uses to generate the sitemap.xml file so I could have my own custom version of it on another url?

Thanks

Steve_Barnes
New Member
20 0 0

Hi there,

We've created an app that allows you to control nofollow and noindex tags, to see more check this out: https://ecommerce.shopify.com/c/shopify-apps/t/new-nofollow-noindex-manager-app-177968

Thanks.

Check out our Shopify SEO Apps on the Store: SEO & Social Meta Manager | NoFollow & NoIndex Manager
Shopify_Craig
Shopify Staff (Retired)
Shopify Staff (Retired)
359 0 27

If you would like to exclude a file from your sitemap file, you can create a metafield (using the Metafields 2 app) with the following properties:

    "namespace": “seo",

    "key": “hidden",

    "value": 1,

    "value_type": "integer"

This will hide the product from the sitemap file.  Keep in mind, this is a pretty advanced use case.  For most people, it's best to put noindex in the meta robots tag on the page itself as this requires just a simple liquid snippet.

 

Craig

Jackson_Cunnin2
Shopify Partner
112 0 13

Is there an easy line of code I can add to our theme template to automatically noindex all products in a certain collection, or with a certain tag?


e.g. I could add a tag 'test' to each new product. Or add each new product in a 'test' collection. 
I don't want to have to do each individual product page manually though (like below)


({% if page.title == "Page Title Here" %}
        <meta name="robots" content="noindex, follow" />
 {% endif %})


I'd rather it all be automatic based on tag or collection instead of doing each individual page.

Thanks
 

Jackson_Cunnin2
Shopify Partner
112 0 13

nvm solved.

 

{% if product.tags contains 'noindex' %}
 <meta name="robots" content="noindex,follow">
{% else %}
       <meta name="robots" content="index,follow">
    {% endif %}

 

Now we just need to add a noindex tag to any product we don't want indexed

emily14433
Excursionist
97 0 5

Who told you?????you can edit the robots.txt file. 101% you can edit it and there are simply two ways; one is edit it in your server side or the second options it to edit in your Google webmaster tool.

Don't panic that once you made you can't edit it.

 

Thanks

Digipro001
Tourist
8 0 4

Hey Steve!  WIll your app work on specific collections?  

Alvin--R
Excursionist
106 1 2

Hi Guys,

 

Are you still in shopify.

 

Why did google always cant index my sitemap?

Can you help me with that?

 

Alvin

Trevor
Community Moderator
Community Moderator
3214 435 743

This is an accepted solution.

Hello!

As of today, June 21st, 2021, we have launched the ability to edit the robot.txt file to give merchants more control over the information that is crawled by search engines. You can learn more about how to edit your robot.txt file through our community post here

Due to the age of the topic, I will be locking this thread and marking it as resolved to help direct anyone that lands on this page. If you have any questions about the new feature, please do not hesitate to create a new post under our "Techincal QA" board.

Trevor | Community Moderator @ Shopify 
 - Was my reply helpful? Click Like to let me know! 
 - Was your question answered? Mark it as an Accepted Solution
 - To learn more visit the Shopify Help Center or the Shopify Blog