Indexed, though blocked by robots.txt https://markandscribe.com/

Shopify Partner
7 0 0

Hi,
I am facing issues with SEO. My site is not getting crawled by google, it says "No information is available for this page.". I have submitted sitemap to google search console but no luck, console is saying "Submitted URL blocked by robots.txt". I am not able to find any solution for this. Please someone help! TIA..

MrunalD
0 Likes
Community Manager
Community Manager
2286 152 312

Hi @mrunalD,

 

Nick here from Shopify. 

 

Just to confirm, does it the error look something like the below screenshot for you? 

 

19-09-hhvb1-mjhlj

If so, then is expected behavior even though the phrasing might sound a little off, strange or like there is a problem. But there more than likely isn't and this is normal. Many pages are intentionally blocked by the robots.txt, this is merely an "alert" that Google has discovered this in case it is unintentional. Everything on the robots.txt is there to help SEO, nothing can be added directly to it and nothing can be removed. It cannot be modified in any way.

 

The reason this is showing is that Google is still using older cached information from your shop or custom domain (if you recently applied their custom domain for example). Because Google is referring to old content from your domain that is no longer relevant. As soon as Google re-indexes the site, the error will be cleared. Furthermore, Google's new robots.txt file is automatically generated by Shopify and can not be edited or deleted (and does not need to be). So your best next step is to re-index your site and see if it changes anything. Shopify has a great guide for this here as well as a YouTube tutorial below: 

 

 

All the best, Nick

Nick | Social Care @ Shopify
 - Was my reply helpful? Click Like to let me know! 
 - Was your question answered? Mark it as an Accepted Solution
 - To learn more visit the Shopify Help Center or the Shopify Blog

0 Likes
Shopify Partner
7 0 0

Hi @Nick ,

Thank you for replying. When I am doing google search, meta description from my home page is not getting displayed on google search. I am getting, "No information is available for this page.". It's fine if robots.txt is blocking cart page, checkout page etc but I am not getting why it's blocking all the pages including home/index page. See the screenshot for more info. 

 

mark and scribe - Google Search.pngScreenshot from 2019-09-24 23-56-05.png

 

I have multiple stores running on shopify platform but didn't face such issue for rest of the stores. I am puzzled how to fix this.

MrunalD
0 Likes
Shopify Partner
7 0 0

Guys, anybody faced same probelm? Any solution to this?

MrunalD
0 Likes
Tourist
9 0 1

Same issue here just found for past 2 days.  Very frustrating.   Seeking Google to "validate fix" but no idea how long it will take them to recrawl. 

0 Likes
Shopify Partner
7 0 0

Hi,

Also check your website for SSL certificate for example:

https://yoursitename.com will work fine but when you will try
https://www.yoursutename.com it will give you the error. This is one of the reason google robot will not be able to crawl the website because it encounters the error. 

MrunalD
0 Likes
Excursionist
51 0 3

I’m having the same issue as MrunalD.

 

It just seems to be for my index page. https://leilacools.com/

 

www redirects to non www and I can’t see a problem there. I really need this fixed but not sure  if it’s a Shopify issue or Google issue.

 

Would appreciate any help!

Leila

 

 

 

 

0 Likes