Indexed, though blocked by robots.txt https://markandscribe.com/

mrunalD
Shopify Partner
7 0 0

Hi,
I am facing issues with SEO. My site is not getting crawled by google, it says "No information is available for this page.". I have submitted sitemap to google search console but no luck, console is saying "Submitted URL blocked by robots.txt". I am not able to find any solution for this. Please someone help! TIA..

MrunalD
0 Likes
Nick
Community Moderator
Community Moderator
3595 292 697

Hi @mrunalD,

 

Nick here from Shopify. 

 

Just to confirm, does it the error look something like the below screenshot for you? 

 

19-09-hhvb1-mjhlj

If so, then is expected behavior even though the phrasing might sound a little off, strange or like there is a problem. But there more than likely isn't and this is normal. Many pages are intentionally blocked by the robots.txt, this is merely an "alert" that Google has discovered this in case it is unintentional. Everything on the robots.txt is there to help SEO, nothing can be added directly to it and nothing can be removed. It cannot be modified in any way.

 

The reason this is showing is that Google is still using older cached information from your shop or custom domain (if you recently applied their custom domain for example). Because Google is referring to old content from your domain that is no longer relevant. As soon as Google re-indexes the site, the error will be cleared. Furthermore, Google's new robots.txt file is automatically generated by Shopify and can not be edited or deleted (and does not need to be). So your best next step is to re-index your site and see if it changes anything. Shopify has a great guide for this here as well as a YouTube tutorial below: 

 

 

All the best, Nick

Nick | Community Moderator @ Shopify 
 - Was my reply helpful? Click Like to let me know! 
 - Was your question answered? Mark it as an Accepted Solution
 - To learn more visit the Shopify Help Center or the Shopify Blog

0 Likes
mrunalD
Shopify Partner
7 0 0

Hi @Nick ,

Thank you for replying. When I am doing google search, meta description from my home page is not getting displayed on google search. I am getting, "No information is available for this page.". It's fine if robots.txt is blocking cart page, checkout page etc but I am not getting why it's blocking all the pages including home/index page. See the screenshot for more info. 

 

mark and scribe - Google Search.pngScreenshot from 2019-09-24 23-56-05.png

 

I have multiple stores running on shopify platform but didn't face such issue for rest of the stores. I am puzzled how to fix this.

MrunalD
0 Likes
mrunalD
Shopify Partner
7 0 0

Guys, anybody faced same probelm? Any solution to this?

MrunalD
0 Likes
Michael_H_1
Excursionist
18 0 21

Same issue here just found for past 2 days.  Very frustrating.   Seeking Google to "validate fix" but no idea how long it will take them to recrawl. 

0 Likes
mrunalD
Shopify Partner
7 0 0

Hi,

Also check your website for SSL certificate for example:

https://yoursitename.com will work fine but when you will try
https://www.yoursutename.com it will give you the error. This is one of the reason google robot will not be able to crawl the website because it encounters the error. 

MrunalD
0 Likes
Smeelah
Trailblazer
264 1 64

I’m having the same issue as MrunalD.

 

It just seems to be for my index page. https://leilacools.com/

 

www redirects to non www and I can’t see a problem there. I really need this fixed but not sure  if it’s a Shopify issue or Google issue.

 

Would appreciate any help!

Leila

 

 

 

 

0 Likes
Sahil_Choujar
Tourist
9 0 2

Hi folks,

 

We are having the exact same issue. And it started just around the same date as it did for @mrunalD.

We've been back and forth with Shopify support and they won't take any responsibility and went as far to say that Google was showing a misleading error.

We've tried all the steps that Shopify support have suggested but even in the latest indexing report our pages cannot be crawled, see attached image.

My client has lost 40% in traffic (and almost the same in revenue) - this is crushing for a small business.


 Screenshot 2019-11-07 10.57.17.png

Hoping they can pay immediate attention to this.

 

Thanks.

0 Likes
Smeelah
Trailblazer
264 1 64

 

 

0 Likes
mrunalD
Shopify Partner
7 0 0

We were having SSL certificate issues also, we had AAAA entry for our domain which were causing issues. We have updated the AAAA entry and our problem has been resolved but still very few pages have been crawled by google. I don't have the exact solution yet but somehow it got resolved. 

MrunalD
0 Likes