Do .atom pages impact SEO and can their creation be stopped?

[sandy786]

are .atom pages affect SEO? Shopify has created several. atom pages. I pulled down links from google search console. I found several . atom page. Will they affect SEO? if yes, then how can I stop Shopify to generate these pages? 

[sandy786]

Hi guys, I need assistance. Is there anyone who can help me?

Please look into this

[ReleafBox]

I have the same issue that just started: https://releaf-box.com/blogs/news.atom Icon appeared in my footer.

Lucas

[Farrell69]

---

@ReleafBox wrote: Employee Connection Insite

I have the same issue that just started: https://releaf-box.com/blogs/news.atom Icon appeared in my footer.

Lucas

---

Thanks for the information, I will try to figure it out for more.

[sandy786]

Hi, Please look into this matter and let me know. As of now I have more than 100 URLs in the Google search console. 

[BenPine]

Likewise, we've also got .atom pages created for our 'collections' pages and am left thinking ... do I redirect them as they don't offer any clear value to a would-be visitor.

[JMalburg]

Hi all, I have a bunch of .Atom pages showing up in our Google Search Console.  How do we get rid of these?  Or should we not worry about them?  Thank you for any help that you can provide!

[CleanCare]

I'm wondering, do I just set them up as a temp removal? Would this fix the problem until Shopify finds a work around? It certainly creates issues with indexing and it will impact SEO if the URL is in a group set that is becoming 'failed' thus halting the index process until fixed. If redirected then that creates a new set to index and then again once the issue is fixed....Has anyone found a best practice for working around this? 

[Fry]

Do you have any updates for this topic?

[Fry]

Maybe it would be a good idea to set an canonical tag for this urls...

[CleanCare]

I did a temp removal because Shopify was telling me this was normal. We all know it's not normal and nobody had a real solution. I was left to just try out the temp removal. It seems to have helped when google decides to crawl the sitemap. Important pages are indexed but this group set (should be indexed) is just removed from the crawler. 

[Fry]

Hi CleanCare, do think the removal does help your site?

[Lisajlb]

I have the same issue and with hundreds of them. Would be helpful if someone from Shopify could give us guidance on this one. I was toying with disallowing them in robots but not sure 

[sandy786]

I really don't know what's going on with this thread, it's been a month. Shopify members didn't reply on it. Can somebody look into this and let me know the best solution for it?  

[IMDB1]

As Shopify Plus member, I've contacted the support and long story short. They are shrugging this off as 'normal operating procedure'. They are directing me towards using Removal tools in Search console. As of right now there's NO way to control .atom file generation or it's content. 

There's a serious exploitable issue with this process and it's allowing hackers from the outside to generate atom files. Here's an example of one on my end:
- /collections/all/Diablo-IV-Item-PC,✔️Best-site✔️⭕X4GM·com⭕✔️Cheap-Price✔️Fast-Delivery✔️Diablo-4-Gold-PS4-PS5,Buy-Diablo-IV-Items,Diablo-4-Items-XBOX-ONE,Diablo-IV-Items-XBOX,d4-Gems-PS4-PS5,Diablo-4-Gold-XBOX-ONE,Buy-d4-Gold,Cheap-Diablo-IV-Gems,diablo-4-paragon-boost,Diablo-4-Items-for-sale, b187.atom

Besides having these unwanted URLS, these atom files are tripping google search console because they are wired wrong, they contain

<link rel="alternate"

What they should contain is:

<link rel="canonical" 

Not 100% sure if Shopify Development is taking this as an issue, let alone a serious one at that. I've logged a feature request, based on my previous experience that could mean this will be addressed anywhere between 6 to 96 months from now. 

[Kay14]

I have exactly the same issue with some of these 'diablo' atom urls created.  Is there a way to remove them?

[sandy786]

I guess this is not going to solve . I have asked several times to support the team, but they didn't help me yet. I don't know how much time they gonna take to resolve this issue. It's been more than several months, this discussion is still going on.

[MD733]

How long has everyone been a Shopify customer? I have been since 2018 and there has been a drastic drop in customer service. 

And even the fact that they have brushed this off to several owners as stated above, they should be monitoring these chats and providing assistance, rather than leaving store owners out to dry and wasting time trying to resolve the problems, when we could be doing more productive things to grow our businesses. 

Even more interesting is WHY would Shopify not assist when this is not an isolated problem and if it is affecting our stores, thus search results are impaired, that means Shopify is making less income also. 

[MD733]

https://support.google.com/webmasters/answer/9689846#make_permanent

[MD733]

Just a thought, have any of you been getting repeated spam emails from people who want to offer their services to do a SEO audit on your site to help your Google rankings? 

I suspect that it could be these types of people who are someone hacking the sites by creating these urls, to block sites from properly appearing in a Google search, thus the reason they offer their assistance. 

There is NO WAY that Shopify is not aware of this, and they should be assisting ASAP. 

[MD733]

Ok, so as we can see there appears to be an ongoing and serious problem here.

1. How are these urls being attached to our sites?

2. How much of a negative impact are these in our sites Google ranking and search results?

3. WHY IS SHOPIFY APPARENTLY IGNORING THE MATTER

So how do we resolve the problem? I recently search online for my business and used almost every key word and my site did not appear in any of the Google search results, where it did previously. 

So I can almost guarantee that these atom urls and others, are affecting our businesses. 

And anyone else have the following urls indexing?

/collections/all/best-dumps-shop,%E2%AD%95Cardx%E3%80%90dot%E3%80%91vip%E2%AD%95dumps-and-pins,vbv-cc-shop,cvv-shop-dumps,buy-vcc-for-paypal-verification,buy-fullz-with-dl-number,buy-cheap-cvv,buy-dumps-cc,card-dumps-with-pin,real-dumps-with-pin,cvv-dumps-2021,%20%20%20655d

/collections/all/buy-dead-fullz

/collections/all/buy-diablo-4-item

/collections/all/%E2%AD%95my

/collections/all/5a44

/collections/all/cc-dumps-for-sale

/collections/all/cheap-diablo-iv-gems

/collections/all/shop-online-without-cvv-code

/collections/all/d2c8

/collections/all/web-facebook-com-hacked

/collections/all/diablo-iv-gems-xbox

/collections/all/plugin

/collections/all/diablo-4-items

/collections/all/analytics

/collections/all/diablo-iv-gems

/collections/all/diablo-4-boosting

/collections/all/best-site-to-buy-cvv-2020

/collections/all/shop-cc-dumps

/collections/all/approved-cvv-shop,%E2%AD%95Cardx%E3%80%90dot%E3%80%91vip%E2%AD%95vcc-for-bing-ads,buy-cvv-fullz,best-site-to-buy-cvv-2020,best-cc-shops,buy-best-dumps,buy-fullz-2020,buy-fullz-with-paypal,free-debit-card-dumps,best-sites-to-buy-cc-dumps,moneybase-cvv-shop,%20%20%20fb6a

/collections/all/centralshop-cvv-shop

/collections/all/online-purchase-without-cvv,%E2%AD%95cardx%E3%80%90dot%E3%80%91vip%E2%AD%95buy-cc-cvv-online,buy-cvv-fullz-online,legit-cc-shop,legit-cvv-sites-2020,buy-credit-card-dumps,dumps-with-pin-shop,valid-dumps-with-pin,best-dumps-shop-2021,buy-fullz-info,cvv-stores-online,%20%20%20bbdb

/collections/all/diablo-iv-gold-pc

/collections/all/d4-item-ps4-ps5

/collections/all/buy-d4-gold

/collections/all/d3-solo-power-leveling

/collections/all/cheap-d4-gems

/collections/all/seo

/collections/all/cheap-d4-items

/collections/all/Diablo-4-Item-XBOX-ONE,%E2%9C%94%EF%B8%8FBest-site%E2%9C%94%EF%B8%8F%E2%AD%95X4GM%C2%B7com%E2%AD%95%E2%9C%94%EF%B8%8FCheap-Price%E2%9C%94%EF%B8%8FFast-Delivery%E2%9C%94%EF%B8%8FDiablo-4-Items-XBOX,Diablo-4-Gold,diablo-4-boost-discord,Diablo-4-Item-PS4-PS5,d4-Item-XBOX,boost-diablo-4,Diablo-IV-Gems,d4-Gold-PS4-PS5,d4-Gold-XBOX-ONE,Buy-Diablo-4-Item,%20%20%207f6d

/collections/all/shop-online-without-cvv,%E2%AD%95Cardx%E3%80%90dot%E3%80%91vip%E2%AD%95buy-vcc-ebay,buy-cvv-fullz,best-cc-shops-2020,cvv2-shop,business-fullz-for-sale,buy-vbv-cc-online,buy-cvv-with-debit-card,cvv-dumps-2021,buy-cvv-online,buy-live-fullz,%20%20%20d2c8

/collections/frontpage

/collections/all/buy-vcc,%E2%AD%95Cardx%E3%80%90dot%E3%80%91vip%E2%AD%95fullz-cc-website,shop-dumps-with-pin,best-sites-to-buy-cc-dumps,buy-fullz-with-paypal,fullz-card,cvv-shop-high-balance,buy-dead-fullz,shop-without-cvv-number,carding-dumps-with-pin,bypass-cvv-shop,%20%20%2092bf

/collections/all/telegram-wp4ya-%E2%AD%95-lifetime 

/collections/all/smelt-gold-tbc

/collections/all/d4-gems-xbox

/collections/all/diablo-4-boost-discord

/collections/all/diablo-4-item-pc

/collections/all/Buy-d4-Gems,%E2%9C%94%EF%B8%8FBest-site%E2%9C%94%EF%B8%8F%E2%AD%95X4GM%C2%B7com%E2%AD%95%E2%9C%94%EF%B8%8FCheap-Price%E2%9C%94%EF%B8%8FFast-Delivery%E2%9C%94%EF%B8%8Fboost-diablo-4,diablo-4-season-boost,d4-Gems-PC,Buy-Diablo-4-Gold,diablo-4-season-24-power-leveling,Diablo-IV-Gold-PS4-PS5,Diablo-4-Items-PS4-PS5,d4-Gems-XBOX,d4-Item,Diablo-4-Item-for-sale,%20%20%206743

/collections/all/telegram-kunghac-%E2%AD%95-hacked-netflix-download

/collections/all/viber-hacked-news

[rhubarbcake]

Yes – my site too – a quick comb over of my indexed URLs show up with the same hacker vomit. 

[rhubarbcake]

This article – albeit for Wordpress and Yoast – actually helps,

https://yoast.com/internal-site-search-spam/

and spotted this on Google Support

https://support.google.com/webmasters/thread/191103351?sjid=3954041795214523557-AP

Now I'm on the search for the n00bie's guide to getting the noindex into my internal search page, if anyone spots a guide any better than this one, keen to hear of it:

https://salt.agency/blog/dealing-with-indexing-issues-on-shopify/

[Fry]

Hi Rhubarbcake,

you can add this to your theme.liquid between the <head> (insert code here) </head>

{% if request.path contains 'search' %}
    <meta name="robots" content="noindex,follow" />
{% endif %} 

The code will set all pages with "search" in the path to noindex. I hope it will help you.

Best Felix

[rhubarbcake]

Awesome, thank you – most grateful! I'm going in!! 🤞

[maplefarmers]

I have the same issue. About 15 that get indexed and 100+ that are not indexed. Mine started in late April and my Google search performance fell off a cliff and is still declining at the same time.

Where is Shopify help?!

[Fry]

My only idea regarding the .atom links is, to overwrite the robots.txt.

However, I'm a little afraid of not always having the latest changes from Shopify in my robots.txt. 😣

Does anyone know if you could somehow copy them into it via liquid?

In this case, would I have the shopify list and can add my own changes...

Best Felix

[thelumosgallery]

Found this link for Customizing Robots.txt and used it to make this addendum to the bottom:
(this line intentionally left blank to separate from the directives above)
User-agent: *
Disallow: *.atom

Disallow: *?pr_prod_strat*

Good thing about the instructions here is it surfaces Shopify's code for creating the Robots.txt so you will not miss updates in the future while adding custom directives to the bottom.

Go through Google Search Console > Pages > Indexing to find search strings and parameter strings to extend this list to clean up your crawl budget.

I'm a technical SEO expert (Brant Soler), my wife is the resident artist at The Lumos Gallery. 😉

[Fry]

perfect, exactly what I was looking for!

[Pivot]

Did you add this to the end of the robots.txt file? I.E. not between one of the 

{%- if ...-%}
..
{%- endif -%}

sections?

[Fry]

Yes, my code looks like this:

# we use Shopify as our ecommerce platform
{%- comment -%}
# Caution! Please read https://help.shopify.com/en/manual/promoting-marketing/seo/editing-robots-txt before proceeding to make changes to this file.
{% endcomment %}
{% for group in robots.default_groups %}
  {{- group.user_agent -}}

  {% for rule in group.rules %}
    {{- rule -}}
  {% endfor %}

  {%- if group.sitemap != blank -%}
    {{ group.sitemap }}
  {%- endif -%}

{% endfor %}

#Shopify Fix 

User-agent: *
Disallow: /*.atom

[Pivot]

Thank you SO MUCH for your response.

[courtandkeith]

Hey Fry,

Please can you tell me if I need to add the whole of your code to my robots.txt file, including the comments? I want to stop wasting crawl budget on .atom files without messing up the rest of my site. Thank you so much

[Fry]

Hey Courtandkeith,

you can copy the whole code. (I would recommend it, because the first part, loads the Shopify robots.txt. ) Of course you can remove the comments - if you like.

My adjustments for the .atom links are only this:

#Shopify Fix 

User-agent: *
Disallow: /*.atom

Best Felix

[ParadoxD]

The proper way would be to add it to the existing group:

# we use Shopify as our ecommerce platform
{%- comment -%}
  # Caution! Please read https://help.shopify.com/en/manual/promoting-marketing/seo/editing-robots-txt before proceeding to make changes to this file.
{%- endcomment -%}
{% for group in robots.default_groups %}
  {{- group.user_agent -}}

  {% for rule in group.rules %}
    {{- rule -}}
  {% endfor %}

  {%- if group.user_agent.value == '*' -%}
    {{ 'Disallow: /*.atom' }}
  {%- endif -%}

  {%- if group.sitemap != blank -%}
    {{ group.sitemap }}
  {%- endif -%}
{% endfor %}

[MD733]

Ok, as we dig deeper down this rabbit hole, we've now been presented with two different ways to create a new robots.txt file to add a rule for blocking the .atom URLS.  Can anyone possibly confirm which coding entry is the best or correct?

OPTION A 

# we use Shopify as our ecommerce platform
{%- comment -%}
# Caution! Please read https://help.shopify.com/en/manual/promoting-marketing/seo/editing-robots-txt before proceeding to make changes to this file.
{%- endcomment -%}
{% for group in robots.default_groups %}
{{- group.user_agent -}}

{% for rule in group.rules %}
{{- rule -}}
{% endfor %}

{%- if group.user_agent.value == '*' -%}
{{ 'Disallow: /*.atom' }}
{%- endif -%}

{%- if group.sitemap != blank -%}
{{ group.sitemap }}
{%- endif -%}
{% endfor %}

VERSUS OPTION B

# we use Shopify as our ecommerce platform
{%- comment -%}
# Caution! Please read https://help.shopify.com/en/manual/promoting-marketing/seo/editing-robots-txt before proceeding to make changes to this file.
{% endcomment %}
{% for group in robots.default_groups %}
{{- group.user_agent -}}

{% for rule in group.rules %}
{{- rule -}}
{% endfor %}

{%- if group.sitemap != blank -%}
{{ group.sitemap }}
{%- endif -%}

{% endfor %}

#Shopify Fix

User-agent: *
Disallow: /*.atom

[ParadoxD]

Don't forget to highlight the {%- endif -%} in the first option. 😉

I'm not sure if you can duplicate user-agent groups in robots.txt or not. Different crawlers may interpret them differently. Since User-agent: * already exists in Shopify's default robots.txt, the safest option would be to add it to the existing group - i.e., option A.

A similar example is given in the docs: https://shopify.dev/docs/themes/seo/robots-txt#add-a-new-rule-to-an-existing-group

[Fry]

Hi @MD733 and @ParadoxD ,

I think Option A would be a good solution. Looks cleaner 😉 But option B should also work.

You can test your robots.txt here: https://www.google.com/webmasters/tools/robots-testing-tool 

Best

Felix

[GeetSoGood]

Hi,

Sorry to bother you but can you please let me know if these pages are important for a website on Shopify: "?pr_prod_strat"

I have hundreds of such pages which are basically copies of my main product page, is that something to do with the functioning/ compatibility of a page on different devices such as Mobiles, Tablets etc?

It would be a huge help if you could tell me if it is wise to disallow them, or if are they important.

[ParadoxD]

A search in the forum revealed this thread which suggests they are urls for product recommendations. There's also this Google support thread that corroborates that. It sounds like you could safely disallow them to save on crawl budget.

[BlackCatSEO]

Thanks !

[MD733]

Has anyone noticed a significant shift in their site traffic?

Since May, there's been a 30% decline in our organic traffic, even though we haven't made any changes. Before May 2023, our organic visits remained consistent for the preceding three years.

Could these links be the reason our website isn't ranking as high on Google search results? Or am I overlooking another factor?

Moreover, given the recent suggestions, what modifications should we make to our theme.liquid code?

If anyone with technical know-how could outline the latest recommended code changes, please specify if they should be made in robots.txt or theme.liquid.

I find it concerning that Shopify hasn't addressed this issue, especially since many stores seem to be affected. It's alarming to think about how many stores might be impacted without even realizing it. I only became aware of this issue by chance when browsing this thread.

I'd also like to extend my gratitude to everyone contributing to this thread. I completely agree with the sentiments expressed earlier – we've spent far too much time on an issue we shouldn't have had to face in the first place.

[ns7]

yes, same here -.-

jackvape.de

[ns7]

same here !
jackvape.de

[CleanCare]

Hello everyone. Growing more frustrated as my removals have all reappeared to my site and inside of GSC. I pressed Shopify for an answer. I was first told that Shopify was working on a fix, then told it was standard operating procedure and they know nothing about it. Then told again that they are working on a fix and that if I wanted these pages removed or hidden to either add it into my robot file or remove them from GSC through the removal tool. The biggest issue I am seeing aside from it just being super annoying and filling my hours with mundane but needed busy work is that I am now getting all sorts of ultra-weird things popping up. Examples of this would be, <h1> tags no longer where they once were. Meta descriptions now have 330 characters instead of the readable 160... It just makes no sense to me. 

[Dams-Care]

Same here.

The second line of the .atom page is something like that 

<feed xml:lang="en" xmlns="http://www.w3.org/2005/Atom" xmlns:opensearch="http://a9.com/-/spec/opensearch/1.1/" xmlns:s="http://jadedpixel.com/-/spec/shopify">

and should include noindex like that

<feed xml:lang="en" xmlns="http://www.w3.org/2005/Atom" indexing:index="no" xmlns:opensearch="http://a9.com/-/spec/opensearch/1.1/" xmlns:s="http://jadedpixel.com/-/spec/shopify"

 or like that

<feed xml:lang="en" xmlns="http://www.w3.org/2005/Atom" name="robots" content="noindex" xmlns:opensearch="http://a9.com/-/spec/opensearch/1.1/" xmlns:s="http://jadedpixel.com/-/spec/shopify">

Anyone from shopify team here?

These .atom pages are used for feed and supposedly help SEM to get updates for fresh content. But it seems that it is not that much the case now? And google decided to index them...

Quick fix : update robots.txt file like that 

# we use Shopify as our ecommerce platform
{% for group in robots.default_groups %}
{{- group.user_agent -}}

{% for rule in group.rules %}
{{ rule }}
{% endfor %}

{%- if group.user_agent.value == '*' -%}
{{ 'Disallow: /*?q=*' }}
{{ 'Disallow: /*?filter*' }}
{{ 'Disallow: /*/sandbox/*' }}
{{ 'Disallow: /*.atom' }}
{{ 'Disallow: /*.oembed' }}

{%- endif -%}

{%- if group.sitemap != blank -%}
{{ group.sitemap }}
{%- endif -%}
{% endfor %}

This should help but the problem is when these atom pages are already indexed. One option is to wait that google desindex by himself these now blocked by robots pages but not sure how efficient it is. We can also manualy ask for noindex in the GSC but that's a pain if we have a lot of pages with this issue.

The best option is to put these atom page in noindex as shown at the begining if shopify is kind enough to do it. Or if anybody is able to help to inject the noindex on these lines. Anyone please?

[courtandkeith]

Hey @Dams-Care, please can you confirm what your robots.txt code achieves? I'm trying to rescue some crawl budget by blocking .atom if they are useless to the crawler. Thanks a lot

[Dams-Care]

Hi,

These will avoid in the order to crawll :

- Search result

- shopify collection filters

- shopify tracking (there was an issue with these url too. It seems resolved but better more than not enough protection)

- The .atom url that are used to facilitate fresh indexing of canonical but get indexed.

- The .oembed url that serve to show a page on another website. I saw one of them indexed as well on GSC so I added it as well.

[Fry]

Hi all,

this is most likely interesting for you too:

https://community.shopify.com/c/technical-q-a/shopify-bug-web-pixels-manager-sandbox/m-p/2119138

Best

Felix

[Pivot]

Thank you!