How to block entire Shopify site from search engines

user1000
Tourist
10 0 1

For now, I don't want my website to be crawled by Google or any other search engine. Could you please let me know how do I block my entire website from the search engine so that Google can't index or crawl it?  Thank you

0 Likes
Olivia
Shopify Staff
Shopify Staff
381 34 88

Hi, @user1000!

Olivia here from the Shopify Team.

Thanks for sharing your question with us. There's a few different ways you can prevent content on your website from being indexed by search engines that I'll outline for you.

Lock your storefront with a password

The quickest way to block your entire site from search engines at once is to enable your built-in storefront password. Head to Admin > Online Store > Preferences > Password Protection > Enable. This won't remove previously indexed search results immediately, but will prevent new content from being updated. As Google crawls the web, old results will become irrelevant and removed. 

Tip: Find and submit your sitemap to force search engines to pull the latest information, or Ask Google to recrawl your URLs.

OR

Hide specific pages from search engines

Another option is to hide certain pages only. You can hide pages that are not included in your robots.txt file by customizing the <head> section of your store's theme.liquid layout file. You need to include some code to noindex particular pages, as such:

{% if handle contains 'page-handle-you-want-to-exclude' %}
<meta name="robots" content="noindex">
{% endif %}

Either of those options will work, but let me know if other questions come up.

Warm regards,

Olivia | Social Care @ Shopify 
 - Was my reply helpful? Click Like to let me know! 
 - Was your question answered? Mark it as an Accepted Solution
 - To learn more visit the Shopify Help Center or the Shopify Blog

0 Likes
user1000
Tourist
10 0 1

Thanks Olivia, I have updated the code under the head section in the theme.liquid file. But, Robot.txt Tester is allowing the page to crawl. Please see the screenshot below and let me know if need to do anything else. thank you

 

Screenshot 2021-02-19 at 10.00.26 PM.png

0 Likes
Olivia
Shopify Staff
Shopify Staff
381 34 88

Thanks for your reply, @user1000!

I took a look at your screenshot, but I'm not entirely sure I understand what you mean.

If you're referring to items showing up as 'disallow' for a robots.txt tester then that would only include items pre-existing in your store's robots.txt file. Since you can't edit this file, the list you are seeing is content that Shopify automatically chooses to block from search engines. This won't include new pages you choose to block, as that isn't part of your robots.txt file.

Let me know if I missed anything.

Olivia | Social Care @ Shopify 
 - Was my reply helpful? Click Like to let me know! 
 - Was your question answered? Mark it as an Accepted Solution
 - To learn more visit the Shopify Help Center or the Shopify Blog

0 Likes