Google is Deindexing all my shopify pages because of the robots.txt file blocking how do i stop this

[mistervape]

I've notcied that my traffic dropped recently and i thought it was the march update.

i've just found out that my pages are being blocked by the robots.txt

i ahve searched for this file in my edit code and cannot find it.

however when i search domain.co.uk/robots.txt it shows.

some of my keywords i was ranking for the pages are being blocked and every day more and more pages are being deindexed and blocked by Excluded by ‘noindex’ tag

i haven't changed anything, but this started 15 days ago and i'm struggling to find the robots.txt in my theme

[shraddha_patel]

Hello,

May I please take a look at the store? Please send store url.

Thank you

[mistervape]

Hi There,
The robots txt is defo blocking some URLS,

i am at a loss though, i can't figure this out.

my URL was ranking on page 1 for keyword "nicotine shots"
url is - https://www.mistervape.co.uk/collections/nicotine-shots

this page is now deindexed by google and it tells me my robots text is blocking it.

I have now everyday more and more pages getting deindexed. traffic is dropping at the same time.

i just can't figure this out.

[Josh_Uebergang]

There's nothing in your robots.txt file that tells me that collection is blocked.

I also used a robots.txt checker. You can use it as well at https://technicalseo.com/tools/robots-txt/
Unless you fixed the robots.txt file, there is something else going on or you're miss-interpreting the Google Search Console report (I presume that's where you're getting your info from.)

[mistervape]

Hi,

This is why i can't work it out.

i was ranking first page for https://www.mistervape.co.uk/collections/nicotine-shots

this now is not indexed, google search console i tested the live url and it says blocked by no index.

i've not seen loads of urls with collections in the name being deindexed

google is taking them all out of the google search now. more daily.


google won't let me request a reindex of the page as they say my robots.txt is blocking it.

[PrintNStuff]

Hei there Josh!

Can you help me figure out why this is happening with my website also and how can I fix it?

https://printnstuff.eu/

 

 

[saim007]

Hi @PrintNStuff You don't need to do anything or it's no need to fix because it is not an error.

These all URLs are not necessary to index in the Google Search it is bad practice so by default Shopify excluded these unnecessary suffix in there robots.txt file so your main URLs are index in the Google Search instead of this multiple URLs.

Now when this flow happen Google Search Console inspect and fetch the whole site and found lots of URLs (which is not require to index in Google Search) so Google Search Console showing those URLs in affected pages section, So you no need to worry your website is totally Google and SEO compatible.

[flareAI]

Hello @mistervape,

Greetings! I am Gina from the flareAI app helping Shopify merchants get $6Million+ in sales from Google Search, on autopilot.

It sounds like your website is being affected by the robots.txt file, which is a file that tells search engine robots which pages or sections of a website to crawl or not crawl. If your pages are being blocked by the robots.txt file, it can result in a drop in traffic and rankings.

If you are unable to find the robots.txt file in your theme, it's possible that it was created or modified by a plugin or app that you have installed on your website. You may want to check if you have any SEO plugins installed that may have created the file or modified it.

If you are still unable to locate the robots.txt file, you can try creating a new one in the root directory of your website. This can be done by creating a plain text file and naming it "robots.txt". You can then add the necessary instructions for search engine robots to crawl or not crawl specific pages or sections of your website.

It's also important to check if any pages on your website have a "noindex" tag, which can also cause pages to be excluded from search engine results. You may want to review your website's meta tags and ensure that none of them contain the "noindex" tag.

Hope this resolves the problem.

Gina

[saim007]

Hello @mistervape 

First of all robots.txt file is not in the Shopify, we can control robots.txt file only in hosting dependable sites which cPanel etc.

So the solution is just copy and paste below code in theme.liquid file before </head> tag your Shopify store - 

<meta name="robots" content="follow, index">

This will allow your whole website to the Google for crawling and indexing.

[Takasa]

Did you ever resolve this? Same thing has happened to our site. We've been trying to fix but nothing is working.