Do .atom pages impact SEO and can their creation be stopped?

sandy786
Tourist
11 0 15

are .atom pages affect SEO? Shopify has created several. atom pages. I pulled down links from google search console. I found several . atom page. Will they affect SEO? if yes, then how can I stop Shopify to generate these pages? 

Replies 64 (64)

sandy786
Tourist
11 0 15

Hi guys, I need assistance. Is there anyone who can help me?

 

Please look into this

ReleafBox
Visitor
1 0 2

I have the same issue that just started: https://releaf-box.com/blogs/news.atom Icon appeared in my footer.

 

Lucas

Farrell69
Excursionist
11 1 3

@ReleafBox wrote: Employee Connection Insite

I have the same issue that just started: https://releaf-box.com/blogs/news.atom Icon appeared in my footer.

 

Lucas


Thanks for the information, I will try to figure it out for more.

sandy786
Tourist
11 0 15

Hi, Please look into this matter and let me know. As of now I have more than 100 URLs in the Google search console. 

BenPine
Tourist
5 0 3

Likewise, we've also got .atom pages created for our 'collections' pages and am left thinking ... do I redirect them as they don't offer any clear value to a would-be visitor.

JMalburg
Tourist
4 0 5

Hi all, I have a bunch of .Atom pages showing up in our Google Search Console.  How do we get rid of these?  Or should we not worry about them?  Thank you for any help that you can provide!

CleanCare
Tourist
4 0 6

I'm wondering, do I just set them up as a temp removal? Would this fix the problem until Shopify finds a work around? It certainly creates issues with indexing and it will impact SEO if the URL is in a group set that is becoming 'failed' thus halting the index process until fixed. If redirected then that creates a new set to index and then again once the issue is fixed....Has anyone found a best practice for working around this? 

 

Fry
Explorer
78 2 38

Do you have any updates for this topic?

Fry
Explorer
78 2 38

Maybe it would be a good idea to set an canonical tag for this urls...

CleanCare
Tourist
4 0 6

I did a temp removal because Shopify was telling me this was normal. We all know it's not normal and nobody had a real solution. I was left to just try out the temp removal. It seems to have helped when google decides to crawl the sitemap. Important pages are indexed but this group set (should be indexed) is just removed from the crawler. 

 

Fry
Explorer
78 2 38

Hi CleanCare, do think the removal does help your site?

Lisajlb
Shopify Partner
10 0 8

I have the same issue and with hundreds of them. Would be helpful if someone from Shopify could give us guidance on this one. I was toying with disallowing them in robots but not sure 

Helping famlies with disabled children
sandy786
Tourist
11 0 15

I really don't know what's going on with this thread, it's been a month. Shopify members didn't reply on it. Can somebody look into this and let me know the best solution for it?  

IMDB1
Visitor
2 0 9

As Shopify Plus member, I've contacted the support and long story short. They are shrugging this off as 'normal operating procedure'. They are directing me towards using Removal tools in Search console. As of right now there's NO way to control .atom file generation or it's content. 

There's a serious exploitable issue with this process and it's allowing hackers from the outside to generate atom files. Here's an example of one on my end:
/collections/all/Diablo-IV-Item-PC,✔️Best-site✔️X4GM·com✔️Cheap-Price✔️Fast-Delivery✔️Diablo-4-Gold-PS4-PS5,Buy-Diablo-IV-Items,Diablo-4-Items-XBOX-ONE,Diablo-IV-Items-XBOX,d4-Gems-PS4-PS5,Diablo-4-Gold-XBOX-ONE,Buy-d4-Gold,Cheap-Diablo-IV-Gems,diablo-4-paragon-boost,Diablo-4-Items-for-sale, b187.atom

Besides having these unwanted URLS, these atom files are tripping google search console because they are wired wrong, they contain

<link rel="alternate"

What they should contain is:

<link rel="canonical" 


Not 100% sure if Shopify Development is taking this as an issue, let alone a serious one at that. I've logged a feature request, based on my previous experience that could mean this will be addressed anywhere between 6 to 96 months from now. 

 

Kay14
Visitor
2 0 0

I have exactly the same issue with some of these 'diablo' atom urls created.  Is there a way to remove them?

sandy786
Tourist
11 0 15

I guess this is not going to solve . I have asked several times to support the team, but they didn't help me yet. I don't know how much time they gonna take to resolve this issue. It's been more than several months, this discussion is still going on.

MD733
Excursionist
16 0 20

How long has everyone been a Shopify customer? I have been since 2018 and there has been a drastic drop in customer service. 

And even the fact that they have brushed this off to several owners as stated above, they should be monitoring these chats and providing assistance, rather than leaving store owners out to dry and wasting time trying to resolve the problems, when we could be doing more productive things to grow our businesses. 

 

Even more interesting is WHY would Shopify not assist when this is not an isolated problem and if it is affecting our stores, thus search results are impaired, that means Shopify is making less income also. 

 

MD733
Excursionist
16 0 20

Just a thought, have any of you been getting repeated spam emails from people who want to offer their services to do a SEO audit on your site to help your Google rankings? 

 

I suspect that it could be these types of people who are someone hacking the sites by creating these urls, to block sites from properly appearing in a Google search, thus the reason they offer their assistance. 

 

There is NO WAY that Shopify is not aware of this, and they should be assisting ASAP. 

 

MD733
Excursionist
16 0 20

Ok, so as we can see there appears to be an ongoing and serious problem here.

 

1. How are these urls being attached to our sites?

2. How much of a negative impact are these in our sites Google ranking and search results?

3. WHY IS SHOPIFY APPARENTLY IGNORING THE MATTER

 

So how do we resolve the problem? I recently search online for my business and used almost every key word and my site did not appear in any of the Google search results, where it did previously. 

 

So I can almost guarantee that these atom urls and others, are affecting our businesses. 

 

And anyone else have the following urls indexing?

 

/collections/all/best-dumps-shop,%E2%AD%95Cardx%E3%80%90dot%E3%80%91vip%E2%AD%95dumps-and-pins,vbv-cc-shop,cvv-shop-dumps,buy-vcc-for-paypal-verification,buy-fullz-with-dl-number,buy-cheap-cvv,buy-dumps-cc,card-dumps-with-pin,real-dumps-with-pin,cvv-dumps-2021,%20%20%20655d

 

/collections/all/buy-dead-fullz

 

/collections/all/buy-diablo-4-item

 

/collections/all/%E2%AD%95my

 

/collections/all/5a44

 

/collections/all/cc-dumps-for-sale

 

/collections/all/cheap-diablo-iv-gems

 

/collections/all/shop-online-without-cvv-code

 

/collections/all/d2c8

 

/collections/all/web-facebook-com-hacked

 

/collections/all/diablo-iv-gems-xbox

 

/collections/all/plugin

 

/collections/all/diablo-4-items

 

/collections/all/analytics

 

/collections/all/diablo-iv-gems

 

/collections/all/diablo-4-boosting

 

/collections/all/best-site-to-buy-cvv-2020

 

/collections/all/shop-cc-dumps

 

/collections/all/approved-cvv-shop,%E2%AD%95Cardx%E3%80%90dot%E3%80%91vip%E2%AD%95vcc-for-bing-ads,buy-cvv-fullz,best-site-to-buy-cvv-2020,best-cc-shops,buy-best-dumps,buy-fullz-2020,buy-fullz-with-paypal,free-debit-card-dumps,best-sites-to-buy-cc-dumps,moneybase-cvv-shop,%20%20%20fb6a

 

/collections/all/centralshop-cvv-shop

 

/collections/all/online-purchase-without-cvv,%E2%AD%95cardx%E3%80%90dot%E3%80%91vip%E2%AD%95buy-cc-cvv-online,buy-cvv-fullz-online,legit-cc-shop,legit-cvv-sites-2020,buy-credit-card-dumps,dumps-with-pin-shop,valid-dumps-with-pin,best-dumps-shop-2021,buy-fullz-info,cvv-stores-online,%20%20%20bbdb

 

/collections/all/diablo-iv-gold-pc

 

/collections/all/d4-item-ps4-ps5

 

/collections/all/buy-d4-gold

 

/collections/all/d3-solo-power-leveling

 

/collections/all/cheap-d4-gems

 

/collections/all/seo

 

/collections/all/cheap-d4-items

 

/collections/all/Diablo-4-Item-XBOX-ONE,%E2%9C%94%EF%B8%8FBest-site%E2%9C%94%EF%B8%8F%E2%AD%95X4GM%C2%B7com%E2%AD%95%E2%9C%94%EF%B8%8FCheap-Price%E2%9C%94%EF%B8%8FFast-Delivery%E2%9C%94%EF%B8%8FDiablo-4-Items-XBOX,Diablo-4-Gold,diablo-4-boost-discord,Diablo-4-Item-PS4-PS5,d4-Item-XBOX,boost-diablo-4,Diablo-IV-Gems,d4-Gold-PS4-PS5,d4-Gold-XBOX-ONE,Buy-Diablo-4-Item,%20%20%207f6d

 

/collections/all/shop-online-without-cvv,%E2%AD%95Cardx%E3%80%90dot%E3%80%91vip%E2%AD%95buy-vcc-ebay,buy-cvv-fullz,best-cc-shops-2020,cvv2-shop,business-fullz-for-sale,buy-vbv-cc-online,buy-cvv-with-debit-card,cvv-dumps-2021,buy-cvv-online,buy-live-fullz,%20%20%20d2c8

 

/collections/frontpage

 

/collections/all/buy-vcc,%E2%AD%95Cardx%E3%80%90dot%E3%80%91vip%E2%AD%95fullz-cc-website,shop-dumps-with-pin,best-sites-to-buy-cc-dumps,buy-fullz-with-paypal,fullz-card,cvv-shop-high-balance,buy-dead-fullz,shop-without-cvv-number,carding-dumps-with-pin,bypass-cvv-shop,%20%20%2092bf

 

/collections/all/telegram-wp4ya-%E2%AD%95-lifetime 

 

/collections/all/smelt-gold-tbc

 

/collections/all/d4-gems-xbox

 

/collections/all/diablo-4-boost-discord

 

/collections/all/diablo-4-item-pc

 

/collections/all/Buy-d4-Gems,%E2%9C%94%EF%B8%8FBest-site%E2%9C%94%EF%B8%8F%E2%AD%95X4GM%C2%B7com%E2%AD%95%E2%9C%94%EF%B8%8FCheap-Price%E2%9C%94%EF%B8%8FFast-Delivery%E2%9C%94%EF%B8%8Fboost-diablo-4,diablo-4-season-boost,d4-Gems-PC,Buy-Diablo-4-Gold,diablo-4-season-24-power-leveling,Diablo-IV-Gold-PS4-PS5,Diablo-4-Items-PS4-PS5,d4-Gems-XBOX,d4-Item,Diablo-4-Item-for-sale,%20%20%206743

 

/collections/all/telegram-kunghac-%E2%AD%95-hacked-netflix-download

 

/collections/all/viber-hacked-news

rhubarbcake
Tourist
7 0 2

Yes – my site too – a quick comb over of my indexed URLs show up with the same hacker vomit. 

rhubarbcake
Tourist
7 0 2

This article – albeit for Wordpress and Yoast – actually helps,

 

https://yoast.com/internal-site-search-spam/

 

and spotted this on Google Support

 

https://support.google.com/webmasters/thread/191103351?sjid=3954041795214523557-AP

 

Now I'm on the search for the n00bie's guide to getting the noindex into my internal search page, if anyone spots a guide any better than this one, keen to hear of it:

 

https://salt.agency/blog/dealing-with-indexing-issues-on-shopify/

 

 

Fry
Explorer
78 2 38

Hi Rhubarbcake,

 

you can add this to your theme.liquid between the <head> (insert code here) </head>

 

{% if request.path contains 'search' %}
    <meta name="robots" content="noindex,follow" />
{% endif %} 

 

The code will set all pages with "search" in the path to noindex. I hope it will help you.

 

Best Felix

rhubarbcake
Tourist
7 0 2

Awesome, thank you – most grateful! I'm going in!! 🤞

maplefarmers
Tourist
3 0 0

I have the same issue. About 15 that get indexed and 100+ that are not indexed. Mine started in late April and my Google search performance fell off a cliff and is still declining at the same time.

 

Where is Shopify help?!

Fry
Explorer
78 2 38

My only idea regarding the .atom links is, to overwrite the robots.txt.

 

However, I'm a little afraid of not always having the latest changes from Shopify in my robots.txt. 😣

Does anyone know if you could somehow copy them into it via liquid?

In this case, would I have the shopify list and can add my own changes...

 

Best Felix

thelumosgallery
Visitor
1 0 3

Found this link for Customizing Robots.txt and used it to make this addendum to the bottom:
(this line intentionally left blank to separate from the directives above)
User-agent: *
Disallow: *.atom

Disallow: *?pr_prod_strat*

 

Good thing about the instructions here is it surfaces Shopify's code for creating the Robots.txt so you will not miss updates in the future while adding custom directives to the bottom.

 

Go through Google Search Console > Pages > Indexing to find search strings and parameter strings to extend this list to clean up your crawl budget.

 

I'm a technical SEO expert (Brant Soler), my wife is the resident artist at The Lumos Gallery. 😉

Fry
Explorer
78 2 38

perfect, exactly what I was looking for!

Pivot
Tourist
4 0 0

Did you add this to the end of the robots.txt file? I.E. not between one of the 

{%- if ...-%}
..
{%- endif -%}

sections?

Fry
Explorer
78 2 38

Yes, my code looks like this:

# we use Shopify as our ecommerce platform
{%- comment -%}
# Caution! Please read https://help.shopify.com/en/manual/promoting-marketing/seo/editing-robots-txt before proceeding to make changes to this file.
{% endcomment %}
{% for group in robots.default_groups %}
  {{- group.user_agent -}}

  {% for rule in group.rules %}
    {{- rule -}}
  {% endfor %}

  {%- if group.sitemap != blank -%}
    {{ group.sitemap }}
  {%- endif -%}

{% endfor %}

#Shopify Fix 

User-agent: *
Disallow: /*.atom
Pivot
Tourist
4 0 0

Thank you SO MUCH for your response.

 

courtandkeith
Visitor
2 0 0

Hey Fry,

 

Please can you tell me if I need to add the whole of your code to my robots.txt file, including the comments? I want to stop wasting crawl budget on .atom files without messing up the rest of my site. Thank you so much

Fry
Explorer
78 2 38

Hey Courtandkeith,

 

you can copy the whole code. (I would recommend it, because the first part, loads the Shopify robots.txt. ) Of course you can remove the comments - if you like.

 

My adjustments for the .atom links are only this:

#Shopify Fix 

User-agent: *
Disallow: /*.atom

Best Felix

 

ParadoxD
Tourist
4 0 3

The proper way would be to add it to the existing group:

 

 

 

# we use Shopify as our ecommerce platform
{%- comment -%}
  # Caution! Please read https://help.shopify.com/en/manual/promoting-marketing/seo/editing-robots-txt before proceeding to make changes to this file.
{%- endcomment -%}
{% for group in robots.default_groups %}
  {{- group.user_agent -}}

  {% for rule in group.rules %}
    {{- rule -}}
  {% endfor %}

  {%- if group.user_agent.value == '*' -%}
    {{ 'Disallow: /*.atom' }}
  {%- endif -%}

  {%- if group.sitemap != blank -%}
    {{ group.sitemap }}
  {%- endif -%}
{% endfor %}

 

 

 

MD733
Excursionist
16 0 20

Ok, as we dig deeper down this rabbit hole, we've now been presented with two different ways to create a new robots.txt file to add a rule for blocking the .atom URLS.  Can anyone possibly confirm which coding entry is the best or correct?


OPTION A 

# we use Shopify as our ecommerce platform
{%- comment -%}
# Caution! Please read https://help.shopify.com/en/manual/promoting-marketing/seo/editing-robots-txt before proceeding to make changes to this file.
{%- endcomment -%}
{% for group in robots.default_groups %}
{{- group.user_agent -}}

{% for rule in group.rules %}
{{- rule -}}
{% endfor %}

{%- if group.user_agent.value == '*' -%}
{{ 'Disallow: /*.atom' }}
{%- endif -%}

{%- if group.sitemap != blank -%}
{{ group.sitemap }}
{%- endif -%}
{% endfor %}

VERSUS OPTION B

 

# we use Shopify as our ecommerce platform
{%- comment -%}
# Caution! Please read https://help.shopify.com/en/manual/promoting-marketing/seo/editing-robots-txt before proceeding to make changes to this file.
{% endcomment %}
{% for group in robots.default_groups %}
{{- group.user_agent -}}

{% for rule in group.rules %}
{{- rule -}}
{% endfor %}

{%- if group.sitemap != blank -%}
{{ group.sitemap }}
{%- endif -%}

{% endfor %}

#Shopify Fix

User-agent: *
Disallow: /*.atom

ParadoxD
Tourist
4 0 3

Don't forget to highlight the {%- endif -%} in the first option. 😉

I'm not sure if you can duplicate user-agent groups in robots.txt or not. Different crawlers may interpret them differently. Since User-agent: * already exists in Shopify's default robots.txt, the safest option would be to add it to the existing group - i.e., option A.

 

A similar example is given in the docs: https://shopify.dev/docs/themes/seo/robots-txt#add-a-new-rule-to-an-existing-group

Fry
Explorer
78 2 38

Hi @MD733 and @ParadoxD ,

 

I think Option A would be a good solution. Looks cleaner 😉 But option B should also work.

 

You can test your robots.txt here: https://www.google.com/webmasters/tools/robots-testing-tool 

Best

Felix

GeetSoGood
New Member
4 0 0

Hi,

 

Sorry to bother you but can you please let me know if these pages are important for a website on Shopify: "?pr_prod_strat"

 

I have hundreds of such pages which are basically copies of my main product page, is that something to do with the functioning/ compatibility of a page on different devices such as Mobiles, Tablets etc?

 

It would be a huge help if you could tell me if it is wise to disallow them, or if are they important.

ParadoxD
Tourist
4 0 3

A search in the forum revealed this thread which suggests they are urls for product recommendations. There's also this Google support thread that corroborates that. It sounds like you could safely disallow them to save on crawl budget.

 

MD733
Excursionist
16 0 20

Has anyone noticed a significant shift in their site traffic?

Since May, there's been a 30% decline in our organic traffic, even though we haven't made any changes. Before May 2023, our organic visits remained consistent for the preceding three years.

Could these links be the reason our website isn't ranking as high on Google search results? Or am I overlooking another factor?

Moreover, given the recent suggestions, what modifications should we make to our theme.liquid code?

If anyone with technical know-how could outline the latest recommended code changes, please specify if they should be made in robots.txt or theme.liquid.

I find it concerning that Shopify hasn't addressed this issue, especially since many stores seem to be affected. It's alarming to think about how many stores might be impacted without even realizing it. I only became aware of this issue by chance when browsing this thread.

I'd also like to extend my gratitude to everyone contributing to this thread. I completely agree with the sentiments expressed earlier – we've spent far too much time on an issue we shouldn't have had to face in the first place.

ns7
Tourist
8 0 1

yes, same here -.-

jackvape.de

ns7
Tourist
8 0 1

same here !
jackvape.de

CleanCare
Tourist
4 0 6

Hello everyone. Growing more frustrated as my removals have all reappeared to my site and inside of GSC. I pressed Shopify for an answer. I was first told that Shopify was working on a fix, then told it was standard operating procedure and they know nothing about it. Then told again that they are working on a fix and that if I wanted these pages removed or hidden to either add it into my robot file or remove them from GSC through the removal tool. The biggest issue I am seeing aside from it just being super annoying and filling my hours with mundane but needed busy work is that I am now getting all sorts of ultra-weird things popping up. Examples of this would be, <h1> tags no longer where they once were. Meta descriptions now have 330 characters instead of the readable 160... It just makes no sense to me. 

Dams-Care
Excursionist
25 0 3

Same here.

The second line of the .atom page is something like that 

<feed xml:lang="en" xmlns="http://www.w3.org/2005/Atom" xmlns:opensearch="http://a9.com/-/spec/opensearch/1.1/" xmlns:s="http://jadedpixel.com/-/spec/shopify">

 

and should include noindex like that

<feed xml:lang="en" xmlns="http://www.w3.org/2005/Atom" indexing:index="no" xmlns:opensearch="http://a9.com/-/spec/opensearch/1.1/" xmlns:s="http://jadedpixel.com/-/spec/shopify"

 or like that

<feed xml:lang="en" xmlns="http://www.w3.org/2005/Atom" name="robots" content="noindex" xmlns:opensearch="http://a9.com/-/spec/opensearch/1.1/" xmlns:s="http://jadedpixel.com/-/spec/shopify">

 

Anyone from shopify team here?

 

These .atom pages are used for feed and supposedly help SEM to get updates for fresh content. But it seems that it is not that much the case now? And google decided to index them...

 

Quick fix : update robots.txt file like that 

 

# we use Shopify as our ecommerce platform
{% for group in robots.default_groups %}
{{- group.user_agent -}}

{% for rule in group.rules %}
{{ rule }}
{% endfor %}

 

{%- if group.user_agent.value == '*' -%}
{{ 'Disallow: /*?q=*' }}
{{ 'Disallow: /*?filter*' }}
{{ 'Disallow: /*/sandbox/*' }}
{{ 'Disallow: /*.atom' }}
{{ 'Disallow: /*.oembed' }}

{%- endif -%}


{%- if group.sitemap != blank -%}
{{ group.sitemap }}
{%- endif -%}
{% endfor %}

 

 

This should help but the problem is when these atom pages are already indexed. One option is to wait that google desindex by himself these now blocked by robots pages but not sure how efficient it is. We can also manualy ask for noindex in the GSC but that's a pain if we have a lot of pages with this issue.

 

The best option is to put these atom page in noindex as shown at the begining if shopify is kind enough to do it. Or if anybody is able to help to inject the noindex on these lines. Anyone please?

courtandkeith
Visitor
2 0 0

Hey @Dams-Care, please can you confirm what your robots.txt code achieves? I'm trying to rescue some crawl budget by blocking .atom if they are useless to the crawler. Thanks a lot

Dams-Care
Excursionist
25 0 3

Hi,

These will avoid in the order to crawll :

- Search result

- shopify collection filters

- shopify tracking (there was an issue with these url too. It seems resolved but better more than not enough protection)

- The .atom url that are used to facilitate fresh indexing of canonical but get indexed.

- The .oembed url that serve to show a page on another website. I saw one of them indexed as well on GSC so I added it as well.

Fry
Explorer
78 2 38
Pivot
Tourist
4 0 0

Thank you!

Patrick_McCabe
Shopify Partner
25 1 25

Hopefully everyone here sees this. I skimmed though and it didn't seem like anyone knew where it was coming from.

I suspect the reason Shopify support has been brushing this off is because this is coming from the content_for_header object. I added the below code to my theme.liquid to test;

 

    {% if content_for_header contains "application/atom+xml" %}
      <!-- Danger! Danger! Danger! Shopify is Dumb! -->
    {% endif %}

 

Sure enough, when I inspected the HTML code of the pages on our site, the comment was present.
So these .atom rel="alternate" URLs are being added by the "required" scripts that Shopify forces you to include in your theme.

I discovered this when I was trying to track down a mysterious rel="canonical" that was present on pages I don't want indexed. I had wrapped the original rel="canonical" in the code below:

 

    {% if request.page_type == 'collection' and collection.handle == 'all' %}
      {% elsif request.page_type == 'collection' and collection.handle == 'vendors' %}
        <meta name="robots" content="noindex">
      {% elsif request.page_type == 'search' %}
        <meta name="robots" content="noindex">
      {% elsif request.page_type == 'product' and product.available <> true %}
        <meta name="robots" content="noindex">
      {%  else %}
        <link rel="canonical" href="{{ canonical_url }}">
    {% endif %}

 

Yet there was a canonical link already present. If you are also dealing with similar mysterious canonical or rel="next" pagination that seems to be coming from nowhere, then this where it is coming from. For us this is a little annoying, as we have backend exclusive products with blank product pages, and Shopify is forcing them to display a .atom feed to search crawlers. I know Google and other search engines will figure things out on their own, but I would still prefer to not have any listed canonical URL for pages I don't want indexed.

Any ideas on what we can do now that the source is identified? Should we try dissecting the content_for_header object with filters to remove the scripts/code we don't want? Would doing so break something or get us in trouble with Shopify?