Do .atom pages impact SEO and can their creation be stopped?

Shopify Discussion
21

Yes – my site too – a quick comb over of my indexed URLs show up with the same hacker vomit.

22

This article – albeit for Wordpress and Yoast – actually helps,

https://yoast.com/internal-site-search-spam/

and spotted this on Google Support

https://support.google.com/webmasters/thread/191103351?sjid=3954041795214523557-AP

Now I’m on the search for the n00bie’s guide to getting the noindex into my internal search page, if anyone spots a guide any better than this one, keen to hear of it:

https://salt.agency/blog/dealing-with-indexing-issues-on-shopify/

1 Like
23

I have the same issue. About 15 that get indexed and 100+ that are not indexed. Mine started in late April and my Google search performance fell off a cliff and is still declining at the same time.

Where is Shopify help?!

24

Hi Rhubarbcake,

you can add this to your theme.liquid between the (insert code here) 

{% if request.path contains 'search' %}
    
{% endif %}

The code will set all pages with “search” in the path to noindex. I hope it will help you.

Best Felix

3 Likes
25

My only idea regarding the .atom links is, to overwrite the robots.txt.

However, I’m a little afraid of not always having the latest changes from Shopify in my robots.txt. :persevering_face:

Does anyone know if you could somehow copy them into it via liquid?

In this case, would I have the shopify list and can add my own changes…

Best Felix

26

Hi CleanCare, do think the removal does help your site?

27

Awesome, thank you – most grateful! I’m going in!! :crossed_fingers:

28

Found this link for Customizing Robots.txt and used it to make this addendum to the bottom:
(this line intentionally left blank to separate from the directives above)
User-agent: *
Disallow: *.atom

Disallow: ?pr_prod_strat

Good thing about the instructions here is it surfaces Shopify’s code for creating the Robots.txt so you will not miss updates in the future while adding custom directives to the bottom.

Go through Google Search Console > Pages > Indexing to find search strings and parameter strings to extend this list to clean up your crawl budget.

I’m a technical SEO expert (Brant Soler), my wife is the resident artist at The Lumos Gallery. :wink:

3 Likes
29

Hello everyone. Growing more frustrated as my removals have all reappeared to my site and inside of GSC. I pressed Shopify for an answer. I was first told that Shopify was working on a fix, then told it was standard operating procedure and they know nothing about it. Then told again that they are working on a fix and that if I wanted these pages removed or hidden to either add it into my robot file or remove them from GSC through the removal tool. The biggest issue I am seeing aside from it just being super annoying and filling my hours with mundane but needed busy work is that I am now getting all sorts of ultra-weird things popping up. Examples of this would be,

tags no longer where they once were. Meta descriptions now have 330 characters instead of the readable 160… It just makes no sense to me.

1 Like
30

perfect, exactly what I was looking for!

31

Did you add this to the end of the robots.txt file? I.E. not between one of the

{%- if …-%}
..
{%- endif -%}

sections?

32

Yes, my code looks like this:

# we use Shopify as our ecommerce platform
{%- comment -%}
# Caution! Please read https://help.shopify.com/en/manual/promoting-marketing/seo/editing-robots-txt before proceeding to make changes to this file.
{% endcomment %}
{% for group in robots.default_groups %}
  {{- group.user_agent -}}

  {% for rule in group.rules %}
    {{- rule -}}
  {% endfor %}

  {%- if group.sitemap != blank -%}
    {{ group.sitemap }}
  {%- endif -%}

{% endfor %}

#Shopify Fix 

User-agent: *
Disallow: /*.atom
2 Likes
33

Thank you SO MUCH for your response.

34

Same here.

The second line of the .atom page is something like that

<feed xml:lang="en" xmlns="http://www.w3.org/2005/Atom" xmlns:opensearch="http://a9.com/-/spec/opensearch/1.1/" xmlns:s="http://jadedpixel.com/-/spec/shopify">

and should include noindex like that

<feed xml:lang="en" xmlns="http://www.w3.org/2005/Atom" indexing:index="no" xmlns:opensearch="http://a9.com/-/spec/opensearch/1.1/" xmlns:s="http://jadedpixel.com/-/spec/shopify"

or like that

Anyone from shopify team here?

These .atom pages are used for feed and supposedly help SEM to get updates for fresh content. But it seems that it is not that much the case now? And google decided to index them…

Quick fix : update robots.txt file like that

we use Shopify as our ecommerce platform

{% for group in robots.default_groups %}
{{- group.user_agent -}}

{% for rule in group.rules %}
{{ rule }}
{% endfor %}

{%- if group.user_agent.value == ‘’ -%}
{{ 'Disallow: /?q=’ }}
{{ 'Disallow: /?filter*’ }}
{{ ‘Disallow: //sandbox/’ }}
{{ ‘Disallow: /.atom’ }}
{{ 'Disallow: /.oembed’ }}

{%- endif -%}

{%- if group.sitemap != blank -%}
{{ group.sitemap }}
{%- endif -%}
{% endfor %}

This should help but the problem is when these atom pages are already indexed. One option is to wait that google desindex by himself these now blocked by robots pages but not sure how efficient it is. We can also manualy ask for noindex in the GSC but that’s a pain if we have a lot of pages with this issue.

The best option is to put these atom page in noindex as shown at the begining if shopify is kind enough to do it. Or if anybody is able to help to inject the noindex on these lines. Anyone please?

35

Hi all,

this is most likely interesting for you too:

https://community.shopify.com/c/technical-q-a/shopify-bug-web-pixels-manager-sandbox/m-p/2119138

Best

Felix

1 Like
36

Thank you!

37

Hey Fry,

Please can you tell me if I need to add the whole of your code to my robots.txt file, including the comments? I want to stop wasting crawl budget on .atom files without messing up the rest of my site. Thank you so much

38

Hey @Dams-Care , please can you confirm what your robots.txt code achieves? I’m trying to rescue some crawl budget by blocking .atom if they are useless to the crawler. Thanks a lot

39

Hi,

These will avoid in the order to crawll :

  • Search result

  • shopify collection filters

  • shopify tracking (there was an issue with these url too. It seems resolved but better more than not enough protection)

  • The .atom url that are used to facilitate fresh indexing of canonical but get indexed.

  • The .oembed url that serve to show a page on another website. I saw one of them indexed as well on GSC so I added it as well.

40

Hey Courtandkeith,

you can copy the whole code. (I would recommend it, because the first part, loads the Shopify robots.txt. ) Of course you can remove the comments - if you like.

My adjustments for the .atom links are only this:

#Shopify Fix 

User-agent: *
Disallow: /*.atom

Best Felix