' Indexed, though blocked by robots.txt' error for cart & account pages

' Indexed, though blocked by robots.txt' error for cart & account pages

Emm_
Shopify Partner
8 0 9

Hello there,

 

Does anybody know/understand why Google keeps on indexing pages that are blocked by robots.txt by default by Shopify (quite rightly so)? In this instance it's a cart page and an account page.

 

This error keeps creeping back regularly for me and I can't figure out why Google decides to override the robots.txt info.

 

Thanks in advance,

Emmanuelle.

 

Screenshot 2024-09-16 at 2.16.10 pm.png

Replies 2 (2)

PaulNewton
Shopify Partner
7201 637 1497

It's probably links on pages that don't have nofollow attributes set on them pointing to pages that are irrelevant for a crawler or search engine to access or display.

Lack of attributes on links is sometimes caused by landing page designs or apps that someone slapped some direct links in.

Contact paull.newton+shopifyforum@gmail.com for the solutions you need


Save time & money ,Ask Questions The Smart Way


Problem Solved? ✔Accept and Like solutions to help future merchants

Answers powered by coffee Thank Paul with a Coffee for more answers or donate to eff.org


naz_ShopiDevs
Shopify Partner
5 0 1

Hi there,

Account, profile, cart etch, this paged not need to fetch the google, for that reason, you can block this page from robots.txt. 

 

The issues you face can happen for various reasons, and here are some:

  • Misconfiguration in the robots.txt file: The instructions provided in the file may be incorrect or not properly formatted.
  • Robots.txt file deletion: The robots.txt file may have been accidentally deleted or removed.
  • Robots.txt file not accessible: The robots.txt file may not be accessible by search engine crawlers. If the server is not configured properly. This could be due to a misconfigured firewall, incorrect file permissions, or other server-side issues.
  • Third-party apps: Some third-party apps can affect the robots.txt file and cause the pages to be indexed despite being blocked by the file.
  • Sitemap: Even if a page is blocked by the robots.txt file, it may still be included in the website’s sitemap, which can lead to the page being indexed by search engines.
  • Links on other pages or dynamic URLs: Despite being blocked by the store’s robots.txt file, some pages may still be crawled and indexed by search engine robots through links on other pages or dynamic URLs.  

naz_ShopiDevs_0-1727075471037.png

Here you see, the cart page is disallowed and it not needs to index. I believe your site is alright and according to the issue you face. 

If you believe this answer is helpful then please mark the answer solved. If you still face issue, then please let us know.  

Found it helpful? Please like and mark the solution that helped you.
Slider Revolution - Create sliders, theme sections, banners, videos, pages, advanced animation, and social feeds.
Essential Grid Gallery - Create photo galleries, video galleries, portfolio galleries, product gallery, collection gallery and more.
EasyDisplay: Product Showcase - Easily display collections, related products, discounts, recently viewed items, and best sellers


Recommnded Shopify Theme– Restaurant, Coffee, Health & Beauty, Kids & Baby, Sunglasses.