ROBOTS.txt

Tourist
8 2 0

Annotation 2019-10-11 131521.jpgWhat does the *+* and *%2B etc mean?

0 Likes
Shopify Staff
Shopify Staff
654 34 76

Hi there @DougAxen!

 

Don here from Shopify. :)

 

These extensions you are seeing there are a kind of 'wild card' to denote that any URL path starting with (for example) /collections will be included in this file no matter what the rest of the URL path is.

 

The +, %2b and %2B symbols all act in this fashion to ensure that the correct content is disallowed and the robots.txt file functions correctly.

 

While it's not possible to edit the robots.txt file to remove disallowed pages, you can 'noindex' further pages you don't want to have crawled by search engines by following the steps given here

 

All the best!

 

 

Regards,

Don

Don | Social Care @ Shopify
 - Was my reply helpful? Click Like to let me know! 
 - Was your question answered? Mark it as an Accepted Solution
 - To learn more visit the Shopify Help Center or the Shopify Blog

0 Likes