robot.txt google how to change crawlning disallowed

Sabine_Mohr
Excursionist
65 0 4

I see robot.txt does this:

Disallow: /collections/*+*
Disallow: /collections/*%2B*
Disallow: /collections/*%2b*
Disallow: /blogs/*+*
Disallow: /blogs/*%2B*
Disallow: /blogs/*%2b*

Now I would like to change it - how would this be possible. Thank you Sabine.

0 Likes
Don
Shopify Staff
Shopify Staff
1291 79 170

Hi there Sabine!

Don here from Shopify!

It's not actually possible to remove blocked pages from the automatically-generated robots.txt file created by your Shopify admin.

What you can do here is to add more pages you would like to have hidden with a little coding work.

All the best!

Regards,
Don

Don | Social Care @ Shopify 
 - Was my reply helpful? Click Like to let me know! 
 - Was your question answered? Mark it as an Accepted Solution
 - To learn more visit the Shopify Help Center or the Shopify Blog

nikki_shah
Tourist
4 0 1

Exact solution Don! Thanks a lot, I got to learn something new

0 Likes
Sabine_Mohr
Excursionist
65 0 4

Hi DON,

why does shopify hide blog and collections?

In this context does it make sense to write a blog from SEO view?

 

Thanks Sabine.

0 Likes
snde
New Member
4 0 0

Hi Don,

We want to know the opposite of what you suggested, i.e. how to unhide (index and follow) a section of the website (say /blogs/)? I read on Google's forum that, that a more stringent setting such as "noindex, nofollow" will overwrite a less stringent setting such as "index, follow". 

If that's the case and the default Shopify setting is "noindex, nofollow" for sections (such as blogs), any code insertion at theme.liquid (such as below) will be ignored. Is that right?

{% if template contains 'search' %}
<meta name="robots" content="noindex">
{% endif %}

 

0 Likes