Needs help with Robots.txt file creation

Solved

Needs help with Robots.txt file creation

BetterDays
Tourist
19 0 1

Hi, I need help with sorting a problem out with my Robots.txt file.

 

I have a brand new site. I got an email from Google Search Console reporting indexing issue. I have attached a screenshot of the issues. 

 

Although I have very basic knowledge of the Shopify platform, I have managed plenty of sites on Wordpress and Webflow and other places, and it included some changes to robots.txt file. However, I am finding a bit lost and frustrated with this. I do know how to access the file following the reading of KB articles and videos. 

 

Can someone show me an example of a standard robots.txt file that I could use on Shopify that would allow the indexing by Google etc? For now I am hoping to copy and paste with some minor adjustments and deal with it properly when I have a bit more knowledge. I just need my site to get indexed without issues.

 

Thank you.

 

GSC Robots TXT Issue.png

 

Accepted Solution (1)

Barked201
Excursionist
22 1 37

This is an accepted solution.

It looks like the Blocked by robots.txt is doing it's job correctly. If you haven't made any adjustments to your code, this is normal.

 

If you want to explore further, do the following:

Click on Online Store

On your Current Theme, click the icon with the three horizontal dots

Select Edit Code

On the left side, click Templates

 

Inside templates, you should see a robots.txt.liquid file. If you don't see it, click Add a New Template and select the robots.txt. It will generate the default robots.txt file for you.

 

The default is:

 

# we use Shopify as our ecommerce platform
{%- comment -%}
# Caution! Please read https://help.shopify.com/en/manual/promoting-marketing/seo/editing-robots-txt before proceeding to make changes to this file.
{% endcomment %}
{% for group in robots.default_groups %}
{{- group.user_agent }}

{%- for rule in group.rules -%}
{{ rule }}
{%- endfor -%}

{%- if group.sitemap != blank -%}
{{ group.sitemap }}
{%- endif -%}
{% endfor %}

 

If you're unfamiliar with how to safely adjust the default code, I would leave it - it seems to be doing it's job correctly. In a new browser if you enter 'www.yourdomain.com/robots.txt' you can see everything that it's blocking.

View solution in original post

Replies 4 (4)

Barked201
Excursionist
22 1 37

This is an accepted solution.

It looks like the Blocked by robots.txt is doing it's job correctly. If you haven't made any adjustments to your code, this is normal.

 

If you want to explore further, do the following:

Click on Online Store

On your Current Theme, click the icon with the three horizontal dots

Select Edit Code

On the left side, click Templates

 

Inside templates, you should see a robots.txt.liquid file. If you don't see it, click Add a New Template and select the robots.txt. It will generate the default robots.txt file for you.

 

The default is:

 

# we use Shopify as our ecommerce platform
{%- comment -%}
# Caution! Please read https://help.shopify.com/en/manual/promoting-marketing/seo/editing-robots-txt before proceeding to make changes to this file.
{% endcomment %}
{% for group in robots.default_groups %}
{{- group.user_agent }}

{%- for rule in group.rules -%}
{{ rule }}
{%- endfor -%}

{%- if group.sitemap != blank -%}
{{ group.sitemap }}
{%- endif -%}
{% endfor %}

 

If you're unfamiliar with how to safely adjust the default code, I would leave it - it seems to be doing it's job correctly. In a new browser if you enter 'www.yourdomain.com/robots.txt' you can see everything that it's blocking.

BetterDays
Tourist
19 0 1

Hi Guys, I need to go over what you both wrote and get back to you.

Thank you.

Emma1134
Visitor
2 0 0

Bonjour, j'ai exactement le même problème.

Pourriez vous s'il vous plait partager votre information pour m'aider à ajuster le code par défaut afin que le robot de google indexe correcte toutes mes pages?

Merci beaucoup pour votre aide.

 

akt-au
Visitor
3 0 0

Hi,

I have created the robots.txt liquid, and requested index for the page from google, but it is still showing the error.

How long does it take for google to recognise the new robots.txt?

 

Thanks

Akt

Screenshot 2025-01-22 at 10.31.47 am.pngScreenshot 2025-01-22 at 10.32.28 am.png