How can I edit robots.txt

Solved
House_Of_Hoye
New Member
1 0 1

I need to edit robots.txt file because, the "https://www.jeremy-hoye.co.uk/collections/unique-engagement-rings"  page not getting link juice value.  How to modify robots file? 

 

Replies 92 (92)
StephenK
Shopify Partner
59 6 27

Well looks like you can now - https://shopify.dev/tutorials/customize-theme-customize-robots-txt-liquid

Some Shopify/Ecommerce related articles - https://medium.com/@stephenkeable
IndieEast
Excursionist
32 0 15

Thanks for the update!  Finally our complaints have been heard and acted upon.

PaulNewton
Shopify Partner
3404 222 651

The feature is rolling out, should have generally availability by monday, so far for all plans.

Notice:  This is an unsupported customization . If you break something you're on your own Shopify support will not help you and will have to pay someone to fix it.

https://help.shopify.com/en/manual/online-store/themes/theme-support#shopify-design-policy 

https://twitter.com/tobi/status/1405981029928779782

Many of the existing directives are there to protect you , be 1000% sure then check again that you understand the syntax of robots.txt before you remove a directive based on a suggestion, especially suggestions that don't understand shopify's url structure and endpoints.

Again this is an unsupported customization. 

This is not like editing a product template that might mess up the layout display there can be real financial costs with a bad edit of robots.txt.liquid sometimes silently bleeding money.

So do not confuse something like 

Disallow: /collections/*+*

as a directive blocking collections as that is not what that syntax does, every character matters and the above single example prevents duplicate tag content.

 

*If the create template method is not working by next week you may need to manually add the file to the theme, also notice this the robots.txt.liquid is in one theme so switching themes may need the contents copied over.

If you know of other robots.txt threads on the forums that need to know let me know the url 

Answers powered by coffee Buy Paul a ☕ Coffee for more answers or donate to eff.org

Problem Solved? ✔Accept and Like solutions to help future merchants

Confused? Busy? Buy a custom solution paull.newton+shopifyforum@gmail.com
IndieEast
Excursionist
32 0 15

Unless someone is an expert, I cannot imagine anyone editing this on their own.  I initially hired someone when they told me it's not allowed, which they thought was strange, but clearly any edits done within our back end by us, has always been our responsibility so this is no different.  We should be allowed to edit whatever we need to as we are paying for the ability to customize our store the way we need to.

StephenK
Shopify Partner
59 6 27

We should be allowed to edit whatever we need to as we are paying for the ability to customize our store the way we need to.

Not sure I agree with this, Shopify is a SaaS platform, so you pay to license the software and for them to host and maintain the technical backend.

If you want control over every aspect of your site then a self hosted and maintained solution rather than SaaS.

Some Shopify/Ecommerce related articles - https://medium.com/@stephenkeable
IndieEast
Excursionist
32 0 15

Sure ok, host and maintain the backend, not control the backend and tell me what I can and can't do with it; what I can or can't change, especially as they are not taking any responsibility.  Other ecomm sites allow editing of the robots.txt, so I don't see what the issue.   Telling everyone after the fact that this is an SaaS platform doesn't help.  They should have mentioned it up front to say, 'btw, you won't be able to make all the edits you want to your back end because we don't allow it'.  But now they do, so the fact that it's an SaaS platform doesn't seem to be an issue anymore.

StephenK
Shopify Partner
59 6 27

Shopify don’t refer to themselves as SaaS explicitly but it is and always has been software as a service.

like anything where you lease, rent or license something, as opposed to building yourself or owning something. There will be restrictions as to what you can and can’t do. Just like leasing a retail location.

Some Shopify/Ecommerce related articles - https://medium.com/@stephenkeable
IndieEast
Excursionist
32 0 15

At least with a lease or license agreement, you have the fine print.  Is this written as a part of the Shopify agreement somewhere that I missed?  Maybe you can point out where?  I'm seriously asking.

StephenK
Shopify Partner
59 6 27

Wow, quite heated for the ability or lack of to edit a robots.txt file ha

Some Shopify/Ecommerce related articles - https://medium.com/@stephenkeable
Grumble
Tourist
8 0 5

Just look at the features in each level. The $80 (?) plan has "Ability to edits robots.txt" - so it's missing from the bottom plan.