Need Urgent Help With Robots.txt

Hellwood_Outfit
Tourist
21 0 1

Hi there...

For some reason the vast majority of my pages (products, collections etc) have got screwed up on Google search. I noticed that they are being blocked by Shopify's Robots.txt for my site. This only became an issue a few months ago but since then visitor number and sales have gone through the floor. How do I go about fixing this? I've trawled through all the data available on Google and can see nothing wrong but something inside the robots.txt. My sitemaps look okay, but all the pages contained within are again being blocked by the robots.txt file. 

I've had a good look around the forum here and it seems pretty clear that Shopify have some sort of issue with allowing their customers to access their own robots.txt file, but if this is still the case, what does Shopify intend to do to solve my issue as it is having a clear impact on my business and if not sorted soon will mean having to jump ship to another ecommerce provider.

Not good!!!

https://hellwoodoutfitters.com
0 Likes
Jason
Shopify Expert
10407 164 2049

What makes you think the pages are being blocked? What's the offending line in the robots file?
If you can give a bit more context the forum members may be able to give better insight. For now, let us start with those questions and go from there.

I jump on these forums to help and share some insights. Not looking to be hired, and not looking for work.

Don't hand out staff invites or give admin password to forum members unless absolutely needed. In most cases the help you need can be handled without that.


★ http://freakdesign.com.au ★
0 Likes
Hellwood_Outfit
Tourist
21 0 1

I did the usual Google search for my own business name and got 

Untitled-1.jpg

So I visited my Google search console to try and work out what the issue was to discover that since the start of August 2094 valid pages that were listed with Google have transformed into error pages. When I looked at why the pages were being marked as errors it clearly states that the submitted urls are being blocked by robots.txt.

Untitled-2.jpg

The urls include my product pages, collection pages, information pages etc. 

I don't think it is unreasonable to make the jump to assuming that my pages are being blocked as that is what it says. As for the offending line in the robots file... I haven't got a clue. Not sure exactly what further context is required, I thought the original post was fairly self explanatory and a look through the forums suggest that I am far from the only one to have this issue, though it seems in their cases too they have either got no assistance at all or got jumped on from the get go.

https://hellwoodoutfitters.com
0 Likes
Hellwood_Outfit
Tourist
21 0 1

??????

https://hellwoodoutfitters.com
0 Likes
Trevor
Community Moderator
Community Moderator
2786 365 562

Hello!

Just a quick update about the robot.txt file that was referenced in this post. As of today, June 21st, 2021, we have launched the ability to edit the robot.txt file to give merchants more control over the information that is crawled by search engines. You can learn more about how to edit your robot.txt file through our community post here

If you have any questions about the new feature, please do not hesitate to create a new post under our "Techincal QA" board.


Trevor | Community Moderator @ Shopify 
 - Was my reply helpful? Click Like to let me know! 
 - Was your question answered? Mark it as an Accepted Solution
 - To learn more visit the Shopify Help Center or the Shopify Blog

0 Likes