All things Shopify and commerce
Hi, I need help with sorting a problem out with my Robots.txt file.
I have a brand new site. I got an email from Google Search Console reporting indexing issue. I have attached a screenshot of the issues.
Although I have very basic knowledge of the Shopify platform, I have managed plenty of sites on Wordpress and Webflow and other places, and it included some changes to robots.txt file. However, I am finding a bit lost and frustrated with this. I do know how to access the file following the reading of KB articles and videos.
Can someone show me an example of a standard robots.txt file that I could use on Shopify that would allow the indexing by Google etc? For now I am hoping to copy and paste with some minor adjustments and deal with it properly when I have a bit more knowledge. I just need my site to get indexed without issues.
Thank you.
Solved! Go to the solution
This is an accepted solution.
It looks like the Blocked by robots.txt is doing it's job correctly. If you haven't made any adjustments to your code, this is normal.
If you want to explore further, do the following:
Click on Online Store
On your Current Theme, click the icon with the three horizontal dots
Select Edit Code
On the left side, click Templates
Inside templates, you should see a robots.txt.liquid file. If you don't see it, click Add a New Template and select the robots.txt. It will generate the default robots.txt file for you.
The default is:
# we use Shopify as our ecommerce platform
{%- comment -%}
# Caution! Please read https://help.shopify.com/en/manual/promoting-marketing/seo/editing-robots-txt before proceeding to make changes to this file.
{% endcomment %}
{% for group in robots.default_groups %}
{{- group.user_agent }}
{%- for rule in group.rules -%}
{{ rule }}
{%- endfor -%}
{%- if group.sitemap != blank -%}
{{ group.sitemap }}
{%- endif -%}
{% endfor %}
If you're unfamiliar with how to safely adjust the default code, I would leave it - it seems to be doing it's job correctly. In a new browser if you enter 'www.yourdomain.com/robots.txt' you can see everything that it's blocking.
This is an accepted solution.
It looks like the Blocked by robots.txt is doing it's job correctly. If you haven't made any adjustments to your code, this is normal.
If you want to explore further, do the following:
Click on Online Store
On your Current Theme, click the icon with the three horizontal dots
Select Edit Code
On the left side, click Templates
Inside templates, you should see a robots.txt.liquid file. If you don't see it, click Add a New Template and select the robots.txt. It will generate the default robots.txt file for you.
The default is:
# we use Shopify as our ecommerce platform
{%- comment -%}
# Caution! Please read https://help.shopify.com/en/manual/promoting-marketing/seo/editing-robots-txt before proceeding to make changes to this file.
{% endcomment %}
{% for group in robots.default_groups %}
{{- group.user_agent }}
{%- for rule in group.rules -%}
{{ rule }}
{%- endfor -%}
{%- if group.sitemap != blank -%}
{{ group.sitemap }}
{%- endif -%}
{% endfor %}
If you're unfamiliar with how to safely adjust the default code, I would leave it - it seems to be doing it's job correctly. In a new browser if you enter 'www.yourdomain.com/robots.txt' you can see everything that it's blocking.
Hi Guys, I need to go over what you both wrote and get back to you.
Thank you.
Bonjour, j'ai exactement le même problème.
Pourriez vous s'il vous plait partager votre information pour m'aider à ajuster le code par défaut afin que le robot de google indexe correcte toutes mes pages?
Merci beaucoup pour votre aide.
Hi,
I have created the robots.txt liquid, and requested index for the page from google, but it is still showing the error.
How long does it take for google to recognise the new robots.txt?
Thanks
Akt
Learn how to expand your operations internationally with Shopify Academy’s learning path...
By Shopify Feb 4, 2025Hey Community, happy February! Looking back to January, we kicked off the year with 8....
By JasonH Feb 3, 2025Expand into selling wholesale with Shopify Academy’s learning path, B2B on Shopify: Lau...
By Shopify Jan 28, 2025