Indexed, though blocked by robots.txt - - ? Big Deal?

New Member
3 0 0

Hi there, 

I have received a handful of "valid but warning" notifications from my Google Search Consul. These 5 warnings are all "Indexed, though blocked by robots.txt" issues. 

I'm not a techy or code person at all and no longer have a web person to help me. When I read the learn more section about this stuff its like reading a foreign language.

Anyways... through my research, it appears everyone recommends that you edit the robot.txt file (whatever that is) but then I discovered that you can't do this if you have a Shopify site...

So what I'm wondering.. do these warnings even matter? What concerns me is that they involve important pages (or at least what I would describe as important)? 

ex. https://sparklerockpop.com/cart

https://sparklerockpop.com/comments

https://sparklerockpop.com/account

https://sparklerockpop.com/account/login

Appreciate any feedback on this. Thanks in advance!

0 Likes
Shopify Staff (Retired)
Shopify Staff (Retired)
33 0 5

Hi there,

My name is Evan, I’m a Guru here with Shopify.

It's always best to err on the side of caution when you get those messages, so it' was good that you asked! In this case, you actually have nothing to worry about. Occasionally Google will send out email warnings just like this, that sound like there is an issue. But it's actually supposed to happen!

Pages are intentionally blocked by the robot.txt to protect your SEO. You'll notice that one of the pages that's blocked is the cart. You wouldn't want Google giving a link to a customer that takes them to an empty cart! We actually have some more information on the Robots.txt here.

While you can't edit the Txt. file it's a good idea to know what it contains! So it's forth taking a look just so you know. It's also good to know incase you want to hide something else, for example down the line maybe you get a WholeSale section and don't want that easily reached.

To hide pages from search engines:

  1. Click the Actions menu, then click Edit code.

  2. Click the theme.liquid layout file.

  3. To exclude the search template, paste the following code in the <head> section:

    {% if template contains 'search' %}
    <meta name="robots" content="noindex">
    {% endif %}

     

  4. To exclude a specific page, paste the following code in the <head> section:
    {% if handle contains 'page-handle-you-want-to-exclude' %}
    <meta name="robots" content="noindex">
    {% endif %}

     

Then be should be sure to replace page-handle-you-want-to-exclude with the correct page handle.

5. Click Save!

While we're on the topic of SEO and business. How is your business doing, do you have any additional SEO marketing goals, or do you need a hand with any other SEO material? If you have any other questions or concerns, please don't hesitate to let me know! I'll be more than happy to help!

Evan | Shopify Guru

2 Likes
Tourist
9 0 4

HI Evan, 

 

I have just been reading through this feed as I have received two e-mails from Google Search Console saying that my 'search' page on my website has been 'Indexed, though blocked by robots.txt'. Am I correct in assuming that this is not a problem, and that I can just leave it as I don't want this page showing up on search engines?

 

Thanks in advance!

 

Sophie 

0 Likes
Highlighted
Tourist
12 0 1

I'd also like confirmation from someone that this is a non-issue.

 

The first Shopify response above says it isn't a problem, but then documents a solution...

 

 

0 Likes

Hey Sparkle_Rock

 

I work with a good amount of Shopify sites so I should be able to help. Short answer: It's not a huge deal and there's not much you can change it. Google is somehow finding those URLs but you cannot adjust the robots.txt.

 

Overall, I'd ignore that message and focus on other things that are going to improve your SEO such as improving content quality. 

0 Likes