How can I remove a spam page from my website index?

[MVUILL]

Hi! 

I received an alert from google search console saying they excluded a page from indexation, and when I look at the page it's spam publicity: 

URL : https://www.french-address.com/collections/vendors?q=Buy%20FUT%2023%20coins%2C%20Cheap%20FIFA%2023%2...

How can I delete this page, I can't find it anywhere? 

Thx for your help!!

[Shay · Accepted solution]

Hi @MVUILL 

I can definitely understand your concern! Based on the URL you shared it looks like this might be a collection created within your admin or possible from the new Shopify Collabs service. 

The "spammy" content of that page you shared is what appears to be the collection page title. If you still cannot find this page within your store admin please reach out to our live support team for additional help with this. 

Please know that our theme and technical support team may be limited in what they can do depending on where/how this page was created. We generally can only support theme edits on our own in-house themes or technical issues that resolve around the Shopify platform itself. That doesn't mean they won't do everything they can to help you get this resolved! 

To contact live support please follow this link: Contact Support - Shopify Help Center.

[Shay · Accepted solution]

Hi @MVUILL 

I can definitely understand your concern! Based on the URL you shared it looks like this might be a collection created within your admin or possible from the new Shopify Collabs service. 

The "spammy" content of that page you shared is what appears to be the collection page title. If you still cannot find this page within your store admin please reach out to our live support team for additional help with this. 

Please know that our theme and technical support team may be limited in what they can do depending on where/how this page was created. We generally can only support theme edits on our own in-house themes or technical issues that resolve around the Shopify platform itself. That doesn't mean they won't do everything they can to help you get this resolved! 

To contact live support please follow this link: Contact Support - Shopify Help Center.

[casetron]

All of a sudden I am logged out of my account to which it says unable to access my site and to contact the store administrator. I am the store administrator. I am a paying customer yet it keeps saying this !

[Maxime_Breton_V]

I am also facing the same issue. As other have mentioned, this issue can only be fixed on Shopify's end and should be fixed as soon as possible.

Everyone should note that this issue does have a negative impact on SEO. Once the spam links started in December 2022, my entire website stopped being shown by Bing search. (As of now, Google search seems unaffected.)

Not a single page of my website is shown anymore on Bing (I cannot even search for my homepage). I used to get around 1000 organic clicks from Bing per month, but now I am getting absolutely 0.

I have implemented the solutions mentioned in this thread and the other thread. A "noindex" tag was added to all the vendor pages. I used the Removal tool to block all "URL/collections/vendors?q=" spam pages.

Although I am speaking for myself, I am sure others have worked very hard for their SEO over many years. It is deeply concerning that all this work is put into jeopardy for something that should easily be fixable by Shopify.

I went from 141 indexed pages to 364,235 indexed pages over the span of one week. Since implementing the "noindex" solution, over 2 million pages have been blocked from indexing.

I can only assume others being hit by this issue have millions of spam pages indexed if they are not quickly made aware of the problem.

I sincerely hope Shopify manages to fix vendor pages to put an end to this issue. We can only try to mitigate the damage as best as we can until Shopify intervenes.

[Egle]

Our page went from 300+ pages to 15.5 million ...........

[DaveSweetCures]

IMO it's a simple fix for Shopify. The vendor area should be an optional extra and controlled with a ON / OFF switch in the stores settings. If switched off then a 404 page should served for any URLs containing /vendor/ 

We've been lucky with Bing in that we applied some of the changes mentioned previously before they really picked up on it. We spiked at 2.7k Pages, around 2k of them being SPAM related to this issue. And since the changes these pages are gradually being removed from their index. 

Google, we saw them pick up and index 367k pages, we made the changes and the not indexed pages hit 2 million. The not indexed pages remains at 2 million, however we are starting to see the indexed pages reducing, we're now on 300k according to Search Console - I always feel Search Console is delayed so hopefully this number is lower.

Only thing we can do is keep commenting on here and encourage others to do so until a proper fix is put in place. 

[Italia-Straps]

This is FYI for the group struggling with this SPAM issue…

I was doing some research over on the Google Search Central Help Community. Apparently, this issue is quite widespread and also is impacting Wordpress sites in addition to Shopify stores.

Once of the threads over there had a link to this article which may be helpful: https://yoast.com/internal-site-search-spam/

The phenomenon seems to be called “internal site search spam”.
The Yoast article is interesting in that is describes what many Shopify store owners are seeing. 

Additionally, the article states “Yoast SEO automatically applies a noindex directive to your search results page, which keeps these URLs out of Google.”

I noticed there is a version of this tool for Shopify. Could this be a possible solution?

I am not familiar with Yoast or endorsing it but wanted to pass on the info.

And… is the noindex directive something Shopify could implement on their end?

Thanks

[Mont]

Hi Shay, i have just discovered same issue.  /collections/vendors?q= Then a bunch of casino references….google is NOT impressed, my ranking is dropping ,sales have stopped. It started end of dec. But here I see others had similar problem last fall. Any remedies? Thank you

[Maxime_Breton_V]

A new exploit seems to have been discovered by the spammers (related to Web Pixels).

Someone more technical than me will be able to understand the logic behind it.

Since March 6th 2023, I have had hundreds of new pages being indexed on Google.

The URLs looks something like this : https://www.mydomain.ca/wpm@0.0.242@9a764525wd403f451p960ae872m43ec93c7/sandbox/blogs/blog-name/blog-article-name 

When I try to follow the link, the pages simply do not exist. I do not understand why google is indexing these pages. I do not understand why spammers are trying to index these pages either.

Ayone else having this issue or have any insight on what to do to solve it?

Thank you in advance!

[shadi1]

This is actually shopify not spammers , its something there doing check out this thread:

https://community.shopify.com/c/technical-q-a/shopify-bug-web-pixels-manager-sandbox/m-p/1982190#M12...

I m very frustrated with it, for some reason blocking the url using robot.txt is not working and its getting indexed as block by robot but indexed.

Shopify should be fixing ASAP

[Maxime_Breton_V]

Thank you for the information, I will look into the other thread!

[Mont]

This is unbelievable. They are asleep at the wheel. I feel helpless. Nov dec had best sales since the supply chain fiasco. Now zip since the holidays. Google wont even run my ads, now I understand why at least. If it wasn’t for google console messages and everyone here I’d still be in the dark. But now what. Known issue for quite some time so don’t have much confidence in Shopify anymore. 

[Greg-Bernhardt]

@Maxime_Breton_V a fix for the web pixel issue is currently rolling out. As with the vendor and search spam fixes, it will take some weeks for Google to revist those pages and unindex them.

@Mont can you paste a few of the vendor spam links that you're seeing in GSC with dates within the last couple weeks? Those URLs should be 404.

[shadi1]

So i should just not bother with no index and all of that?

[Greg-Bernhardt]

Correct, can you paste a few of those pixel URLs in here and we can see if they 404

[shadi1]

yeah i got close to 400 over night, i been keeping an eye on my indexed pages, so this is very new.

[Greg-Bernhardt]

@Anonymous can you post the text URLs here do I can check the status codes

[shadi1]

Sure

https://xtremedigitalgraphix.com/wpm@0.0.234@1543f512w2542bf6cpa9acea22m3c2685ed/sandbox/products/ca...

[Greg-Bernhardt]

Looks like there is a correct header of noindex. Can you check your robots file to see if you're blocking this URL and if so, unblock it.

[shadi1]

I think this should work my brother helped by putting this code in the robot.txt file to unblock the url.  Hopefully this will take care of the issue.

[Brenden_H]

We have the same issue on our site that @shadi1 has. 

I looked at the Shadi1 code and they have not got the page set to noindex as you say they have. So even if they do unblock this pathway on robots.txt (something that Shopify forced on all of us when it was updated to disallow crawling of any URLs with /wpm@) then it will still get crawled and still get shown in the index.

Why hasn't the same solution been applied to those URLs with /wpm@ as you did to those with /web-pixel-manager@? As whilst not ideal at least with a 404 page and the noindex directive, I can simply request search engines remove the URLs from the index and it will be sorted ina day or two. With crawling being blocked by robots.txt, there is nothing to stop search engines from adding them back to the index again.

Will you be applying the same fix to the wpm@ versions that you did to the web-pixel-manager@ URL’s going forward? Or will you be resolving the issues that are causing these pages to be found in the first place?

[shadi1]

I m going to look deeper into it today , but the shopify staff member stated here that the page is set to no index and all i have to do is undo the blocked wpm@ from the robot.txt.

Honestly i m getting sick of this bs. Every time i start to get my ranking and traffic up something new takes me down.

My blocked by robot text but still indexed count jumped by 1k today. I understand this will take some time, but how is this happening!

Brenden_H is right Shopify we need the same fix that was done to /web-pixel-manager ASAP.

[Maxime_Breton_V]

Hey everyone,

My indexed pages are also shooting up day after day because of the "wpm@"  URLs. This is stressful as I have just been able to get reindexed on Bing after 3 months (following the spam pages exploit).

I am unsure what solution to implement while we wait for Shopify to fix the issue on their end. There seems to be contradicting information being provided.

Can anyone from Shopify give us a temporary fix that we could implement while we wait for the permanent fix on Shopify's end?

For those of us who are not very technical, detailed instructions would be very helpful.

By the way, thank you for you help until now @Greg-Bernhardt.

[Greg-Bernhardt]

We're still internally discussing a solution regarding the pixel pages. I'll report when I have more information.

[Brenden_H]

@Greg-Bernhardt do you have a timeline for when we can expect to have an answer?

[shadi1]

Greg are you sure there no index no follow implemented right now?

[Greg-Bernhardt]

No timeline. Noindex is implemented, but robots is blocking crawling from seeing it atm.

[Brenden_H]

I've attached an image of the source code from the page that @shadi1  shared and it is not set to noindex.

The noindex setting is only showing on the versions of the URL which start with /web-pixels-manager@ address - there are no meta robots directives on URLs starting with /wpm@ as I have checked dozens of them on our site and on the person who shared theirs here. 

[Greg-Bernhardt]

I believe the noindex is sent via header

[Mont]

Hi ...I could be wrong this is all over my head, but I focussed on sitemap.xml , since that is what Google uses to index. I went through support. Wasn't me , somehow my sitemap (products) was revised from

https://vr.supplies/sitemap_products_1.xml  (which was filled with garbage links)

to

https://vr.supplies/sitemap_products_1.xml?from=1680358867017&to=6815972884560 (which is just my product)

Whoever did this ...THANK YOU! seems to be a good solution.  

GO to your domain.xxx/sitemap.xml copy and paste into URL Field and see what's there. Then go to your Google Console and ask Google to re-index it now.

[Greg-Bernhardt]

@Mont I've tried some merchant sites and that first sitemap URL results in a 400 error

[Maelb22]

Hi @Greg-Bernhardt 

Google continues to index "search" spam pages. This is really becoming problematic, and it's starting to impact my SEO (loss of keywords, loss of traffic...).

The solution with "no index" doesn't work. Every day, google indexes new "search" spam pages.

The right solution would be to implement a 404 on these pages. As for the "vendors" problem. It is through this solution that the "vendors" problem has been solved.

What about Shopify, why don't you implement this same solution ?

Be aware that our stores are directly impacted by your failures, and the delay counts.

Thank you for considering my request.

Best regards

[Maxime_Breton_V]

I would just like to confirm that new "search" spam pages are also still being indexed by Google on my end also.

The number of spam pages being indexed keeps going back up.

Is any permanent solution in the works?

[Greg-Bernhardt]

@Maxime_Breton_V can you DM a /search link that is being indexed?

[Brenden_H]

Yes @shadi1, I checked the page you shared and it isn't set to noindex as Shopify have done with the first version of the URL. I don't believe there is a way we can set it as noindex ourselves as these pages are not in the structure available to us to add these directives using Shopify apps.

If you want to take them out of the index temporarily so that search engines are not judging you on this, go to your Google Search Console and on the left-hand side of the screen select "Removals" in the indexing section. Then add this URL as a temporary removal request using the "new request" button: https://xtremedigitalgraphix.com/wpm@

It will then remove anything using that entire pathway from the index. Search Console will take a week or so to catch up but if tomorrow you go onto Google and use the site: search and add that URL to it there shouldn't be anything indexed there. Then you can wait on Shopify to resolve at their end as this removal can last for 3-6 months and will cover all versions of that URL pathway.

[shadi1]

I just found out i got the same issue what is going on? Only few ppl have access to the site, the rest are few collaborators, there last access was 8 month!

Any one else has this issue???

[NEI-Arlene]

 

I'm having the same issue. I spoke with Shopify and they disregarded saying that it is most likely the themes third party and to contact them. However, when looking at other forums and doing a quick google search there are thousands of Shopify accounts affected. 

This is in fact a malware.
I called my domain company and they did confirm this site was generating malware but due to Shopify's limitations on providing file transfer privileges they could not delete or remove the malware and stated Shopify has to do it. 

I'm contacting Shopify again today to see if anything can be done.  

[shadi1]

This is a vicious bot that has effected a lot of stores online, not just shopify . Its not in shopify code or in your theme. Its using a vulnerability in most website venders?q section to create this fake fifa url page you see , which can be done by any one, but what sucks is that this bot is also creating links on another site to this fake url it created on your site. Then google indexes that page and shows it in the search result.

shopify robot.txt is already blocking indexing this page , but google has this stupid policy that says if another website has a link to a page on your site then google will still index that page regardless if your robot.txt is asking not index .

Its all over the internet effecting most of online stores. If you do a google search of fifa 23 coins you will see what i mean.

So far shopify told me all i can do is disavow the website with link to this fake created fifa url on my site.

If you find a solution let us now.

[MVUILL]

What's the process to disavow a website? I'd like to do it too!Thx

[Barked201]

You can disavow all you want, it won't stop this. Every shop is experienced thousand or millions of new indexed pages daily, as you can see across these boards. It's an exploit that Shopify is choosing to ignore rather than address.

The only solution that I've found so far from @Jizo_Inagaki  is to do the following:

Go to the theme.liquid file and enter the following code in the <head> section:

{%- if request.path == '/collections/vendors' and collection.all_products_count == 0 -%}
<meta name="robots" content="noindex">
{%- endif -%}

This should add a 'noindex' tag to the collection/vendors pages being exploited. As your site is crawled in the future, it will remove current pages and stop future ones from being indexed on your site.

Shopify is choosing to ignore this, so hopefully that stops the bleeding for now.

[Jesper_Skaane_B]

@Barked201seems straight forward. how does this difference from adding Disallow: /collections/vendors* in robots? (if it does)

[Barked201]

If you block with robots.txt it will prevent Google from crawling those pages to 'noindex', I believe. My understanding is that if they're already indexed, which most are, and you block with robots.txt, Google won't be able to crawl again to remove them from index.

Adding the <meta name="robots" content="noindex"> code above will allow Google to crawl these pages and adhere to the 'noindex' rules, removing it from index and preventing any others from indexing in the future.

I tested a live URL on my site and it seems to be working - I just have to wait for them all to be crawled again, which can take a while.

[bradm24]

Thanks, we found that this is also working to noindex the vendor pages. What code did you use to stop writing the query into the title tag? I tried this but it ends up breaking the title tags on each page and only prints the url of the page.

{%- if request.path == '/collections/vendors' and collection.all_products_count == 0 -%}
<title>No Results Found</title>
{%- else -%}
<title>{{ seo_title | strip }}</title>
{%- endif -%}

[NEI-Arlene]

Update: I spoke with A Shopify rep yesterday and no luck. 

Shelbee (Shopify)

Nov 2, 2022, 00:02 EDT

Hey Arlene,

Shelbee here from Shopify Support. Just following up from our call we had today in regards to what you have seen in your Google Merchant Centre.

I was able to get some more feedback from our Technical Team and they have provided me with the below information to relay over.

The question mark in a URL (like in this link) signals that everything after will be a parameter (something a store visitor has entered into the site) and are not part of the base URL in the same way a product URL is displayed. Note that a base URL (A URL originating from a product, collection, page, etc) like this one from your store does not contain a ?

The main concern here is about the impact this may have on your SEO. While annoying this page has little to no impact on the store and no action needs to be taken. If you do note a considerable impact on SEO or a large number of these URLs have being identified (large beings dozens or hundreds) then there are steps you can take. However, these also carry a level of risk in terms of SEO.

Using you SEO reporting software you can collect all the bad backlinks into a .txt file and report them via Google's Disavow Tool. (NOTE: the backlinks you need to list will be the referral site address rather than their search term URL.) This is an advanced feature and should only be used with caution. If used incorrectly, this feature can potentially harm your site's performance in Google's search results. We recommend that you only disavow backlinks if you believe that there are a considerable number of spammy, artificial, or low-quality links pointing to your site, and if you are confident that the links are causing issues for you.

The team has assured me there is no solution to this issue that originates from the Shopify Platform. There are fantastic third-party guides to minimizing the impact of backlinks as well, we would recommend looking into this guide on this.

Take care,

Shelbee | Support Advisor | Shopify

So it seems that even though I request for a Disavow the malware already installed is not going to be cleaned or removed by Shopify. 

This is the second time I get the same, "no solution to this issue that originates from the Shopify Platform" & "this page has little to no impact on the store and no action needs to be taken". 

I've already added the following code to my theme in hopes that the vendors won't be crawled by bots and indexed.

{%- if request.path == '/collections/vendors' and collection.all_products_count == 0 -%}
<meta name="robots" content="noindex">
{%- endif -%}

Doesn't change the fact that there is malware installed that can't be removed. 

If anyone has any idea on how to remove malware please let me know. 

Thanks!

-Arlene

[Shay]

Hi everyone! Thank you for sharing the screenshots here and letting me know that this page is being created in more than one store. If you have created a ticket for this already, even if you were previously told there isn't anything more we can do, can you please share that ticket number here with me and also share the URL for this page on your store. If you have not yet created a ticket about this, please reach out to our live support to create one. Feel free to share a link to this forum post as well when doing so.

While I am not able to directly view accounts through the community forums, I feel that we definitely have cause for a deeper investigation by our technical team as to where this page is being created from. There is a high chance that it is being injected by a third party app that hasn't been caught yet. We absolutely want to put a stop to that.  

[NEI-Arlene]

I have 2 tickets created, 33975571 & 33965781.

This is not an issue with any third party app on shopify or theme. 

There are over 220,000 Shopify & NON-Shopify websites affected. 

This is a quick google search for the text that's populating on our websites. 

I've contacted my Domain and they confirmed there is in fact malware installed. 

We need a sweep and removal of malware as Shopify users.

 

[shadi1]

OK i m going to show you that this is a vulnerability with the vendors?q section of any website:

Any one can use this code here is a screenshot of the code used notice how the highlighted in blue section added says "thisisatestforshopify":

Anyone can do this to any website , its not a malware in shopify servers, at least i don't think it is since its effecting a lot of other non shopify stores. This is probably a bot going around doing this to most online stores on the net that have the vendors?q section .

I already asked shopify to see if they can turn off the vendor?q section and there reply was NO since its a vital part of platform.

As i mentioned in my earlier post, the problem is google indexing that page created by the bot and showing it in the search result. Last time i searched on google millions of online stores are effected worldwide.

I tried to modify my robot.txt so that google won't index these pages. But i soon found out that is useless since google has this policy. If another website has a link to that  page google will still index that page on ur site and ignore your robot.txt instruction not to index. Which is what the bot is doing as its creating fake a page in an online store its also creating a link to it.

Google policy is to ignore the site owners request not to index specific pages.

I honestly don't know if this effecting our SEO , if its happening to everyone aren't we all in the same boat?

Using my google search council i only found one site linking to this created fifa url , i m going to disavow it and see if that does anything!

[Shay · Accepted solution]

Thank you @NEI-Arlene for that additional information and your open tickets about this situation. I have connected with our security team about this concern and I can share some insight and best next steps to get this resolved. 

When reviewing these links, it is important to understand how they function and how they were initially created.

Example URL: https://www.yourstoreurlhere.com/collections/vendors?q=test 

The "?q=" in the URL is sending a search query to the website in the first part of the url structure and it is searching for whatever is placed after the URL. 

If you went to your own website and added "/collections/vendors?q=test" to the end of your store address and hit enter, you would see a page show up with the page title being "test" and no products found. These URLs can be made by anyone and will generally work on any website with a search function. 

Malicious external websites will create these empty backlinks to store URLs to help promote their services or products by using the search query on the website to generate a page with their product details as the title. The page itself doesn't exist independently, it only exists as part of a search result on the website being targeted. 

How to disavow these backlinks from Google.

Using a SEO reporting software you can collect all the bad backlinks into a .txt file and report them via Google's Disavow Tool. Full steps on how to do this are in the link. (NOTE: the backlinks you need to list will be the referral site address rather than the search term URL.)

Please note the following warning on Google Search Console:

This is an advanced feature and should only be used with caution. If used incorrectly, this feature can potentially harm your site's performance in Google's search results. We recommend that you only disavow backlinks if you believe that there are a considerable number of spammy, artificial, or low-quality links pointing to your site, and if you are confident that the links are causing issues for you.

Also, a great resource to learn more about how backlinks work: How to Stop Spam Backlinks from Ruining Your Google Reputation.

If you have any concerns about reporting these backlinks to Google or researching more information on your website's SEO then I recommend hiring an expert from our expert marketplace that specialize in this field and can assist you further: Hire Shopify Experts, developers, designers and freelancers.

Edited to add: There is another forum thread in the community here with a possible solution for stopping these kinds of backlinks from working:  Solved: Re: Has my site been hacked? 

[Allan-EP]

Chiming in to say we have a client experiencing this as well. 

To Shopify staff: this is a bug. When you request a path of /collections/something?q=foo, {{ collection.title }} should not be outputting "foo" because the collection object should be nil, assuming collection "Something" doesn't exist. In fact, it should 404. But in the case of "Vendors" it loads as if the collection exists and passes the value of q into the title property. 

Passing this off as "any site will do this" is simply not true. While yes, you can throw any parameter in a URL, the server has to actually accept the parameter and do something with it. In the case of "Vendors", Shopify should be returning a 404 (not a 200) when no such collection exists. All the workarounds suggested in this thread are bandaids and don't address the root problem.

[jcop24]

Interesting. I am having the same issue but mine is "euromillion" & lottery-type stuff and not FIFA. The majority of it was in Chinese so I needed to use google translate to see what it says. If you search "euromillion shopify" in google, numerous sites pull up but you can tell they are all malware on normal sites. 

It is also in my venders?q section.

I haven't submitted a ticket yet because I was just researching it and found this thread. 

 @Shay @shadi1 @Greg-Bernhardt