Shopify and Robots.txt

Solved
SeanDawes
Shopify Partner
149 0 3

Curious if there is any way to access the robots.txt file for shopify stores?

Saw an old post in 2011 with a one word answer of "no". Was hoping shopify has realized the importance of this for any real store and if there is any progress on getting this as a feature?

 

 

Former SEO of Inc Top 500 Retailer AmericanMuscle.com who Co-Founded Online Retail Marketing Company, Rocket Dove (http://www.rocketdove.com) where we drive millions in revenue across various ecommerce industries.
0 Likes
HunkyBill
Shopify Expert
4509 46 492

Why is it so important for you to edit the file? If you know the existing file generated automatically is somehow deficient could you please share with the community how exactly you'd edit that file and in turn make a shop one more real than they currently are with the existing scheme?

Be nice to know.. that's all.. I am always curious to learn something new from experts outside my domain.

Thanks!

Custom Shopify Apps built just for you! hunkybill@gmail.com http://www.resistorsoftware.com
0 Likes
Nate_Shivar
Shopify Expert
7 0 0

I haven't tested it exactly, but you should just be able to write out a robots.txt file in NotePad, then go to Themes --> Template Editor --> Assets --> Add new asset, then upload your custom robots.txt

Test it by going to [yourdomain]/robots.txt

And ditto hunkybill - are you looking to block certain directories, or what?

I help online stores with Google AdWords and pay-per-click advertising at http://www.ShivarWeb.com, and love Shopify (former store owner!).
0 Likes
SeanDawes
Shopify Partner
149 0 3

I would like to edit the file to start blocking certain urls that just should not be indexed but are such as constraints on urls. 

Sorry for the delay. I did not get an email notification on this and just remembered to check

Former SEO of Inc Top 500 Retailer AmericanMuscle.com who Co-Founded Online Retail Marketing Company, Rocket Dove (http://www.rocketdove.com) where we drive millions in revenue across various ecommerce industries.
0 Likes
rickydazla
Shopify Expert
690 0 73

Wrap your <head> around this:

{% if page_title contains 'Whateva' %}<meta name="robots" content="noindex">{% endif %}
I'm a million different people
standoutd
Navigator
1135 0 128

We use a metafield that we set to True. Manage it with the MetaFields app. Works great.

 

http://www.StandoutDesigns.com ::: Solid Wood TV Furniture for Enthusiasts. Made in USA.
0 Likes
Czarto
Shopify Partner
66 2 19

What about if you want to intergrate Wistia.com video SEO?  Usually they want to include a sitemap link similar to

Sitemap: http://app.wistia.com/sitemaps/9369.xml

Into your robots.txt...  Having a hard time figuring out how to do this with Shopify.

Digital Marketing @ Manitobah Mukluks
Shopify Digital Marketing Blog: business.czarto.com
0 Likes
Trevor
Community Moderator
Community Moderator
2543 327 518

This is an accepted solution.

Hello!

As of today, June 21st, 2021, we have launched the ability to edit the robot.txt file to give merchants more control over the information that is crawled by search engines. You can learn more about how to edit your robot.txt file through our community post here

Due to the age of the topic, I will be locking this thread and marking it as resolved to help direct anyone that lands on this page. If you have any questions about the new feature, please do not hesitate to create a new post under our "Techincal QA" board.

Trevor | Community Moderator @ Shopify 
 - Was my reply helpful? Click Like to let me know! 
 - Was your question answered? Mark it as an Accepted Solution
 - To learn more visit the Shopify Help Center or the Shopify Blog

0 Likes