Discuss and resolve questions on Liquid, JavaScript, themes, sales channels, and site speed enhancements.
Shopify has a new bug - they are indexing tons of useless pages in Google again. Is there any way to stop this from happening? The code seems to only be editable by shopify since it's in the content_for_header.
Do a google search for:
inurl:/web-pixels-manager@
(https://www.google.com/search?q=inurl%3A%2Fweb-pixels-manager%40)
Each site has multiple versions of this page indexed. Peets.com for example has 20.
The pages look blank, but include something like this in the code:
<!DOCTYPE html>
<html>
<head>
<meta charset="utf-8">
<title>Web Pixels Manager Sandbox</title>
<script id="web-pixels-manager-setup">(function e(e,n,a,o,t){e&&(window.Shopify=window.Shopify||{},window.Shopify.analytics=window.Shopify.analytics||{},window.Shopify.analytics.replayQueue=[],window.Shopify.analytics.publish=function(e,n,a){window.Shopify.analytics.replayQueue.push([e,n,a])});var r,i,s,l,d,c,p,u,f=a+"/"+o+"."+function(){var e="legacy",n="unknown",a=null,o=navigator.userAgent.match(/(Firefox|Chrome)\/(\d+)/i),t=navigator.userAgent.match(/(Edg)\/(\d+)/i),r=navigator.userAgent.match(/(Version)\/(\d+)(.+)(Safari)\/(\d+)/i);r?(n="safari",a=parseInt(r[2],10)):t?(n="edge",a=parseInt(t[2],10)):o&&(n=o[1].toLocaleLowerCase(),a=parseInt(o[2],10));var i={chrome:60,firefox:55,safari:11,edge:80}[n];return void 0!==i&&null!==a&&i<=a&&(e="modern"),e}()+".js";r={src:f,async:!0,onload:function(){if(e){var a=window.webPixelsManager.init(e);n(a),window.Shopify.analytics.replayQueue.forEach((function(e){a.publishCustomEvent(e[0],e[1],e[2])})),window.Shopify.analytics.replayQueue=[],window.Shopify.analytics.publish=a.publishCustomEvent}},onerror:function(){var n=(e.storefrontBaseUrl?e.storefrontBaseUrl.replace(/\/$/,""):self.location.origin)+"/.well-known/shopify/monorail/unstable/produce_batch",a=JSON.stringify({metadata:{event_sent_at_ms:(new Date).getTime()},events:[{schema_id:"web_pixels_manager_load/2.0",payload:{version:t||"latest",page_url:self.location.href,status:"failed",error_msg:f+" has failed to load"},metadata:{event_created_at_ms:(new Date).getTime()}}]});try{if(self.navigator.sendBeacon.bind(self.navigator)(n,a))return!0}catch(e){}const o=new XMLHttpRequest;try{return o.open("POST",n,!0),o.setRequestHeader("Content-Type","text/plain"),o.send(a),!0}catch(e){console&&console.warn&&console.warn("[Web Pixels Manager] Got an unhandled error while logging a load error.")}return!1}},i=document.createElement("script"),s=r.src,l=r.async||!0,d=r.onload,c=r.onerror,p=document.head,u=document.body,i.async=l,i.src=s,d&&i.addEventListener("load",d),c&&i.addEventListener("error",c),p?p.appendChild(i):u?u.appendChild(i):console.error("Did not find a head or body element to append the script")})(null,null,"https://cdn.shopify.com/shopifycloud/web-pixels-manager/0.0.186","sandbox","0.0.186");</script>
</head>
<body></body>
</html>
Solved! Go to the solution
This is an accepted solution.
Hello everyone.
Thank you for your patience while we worked towards providing a fix to this issue and building out an FAQ that addresses the primary concerns and questions from this thread. You can find our FAQ on the Community Blog.
We welcome you to continue the conversation in the blog section of the blog post with any valid questions that aren't already answered by the FAQ. We will monitor the comments section for any valid feedback on how this change may impact Shopify stores and will actively remove or edit any comments that spread misinformation or speculation on the issue.
On that, this thread contains some misinformation and speculation on the issue and how it may impact ones online store. For this reason, we will no longer be monitoring this thread, but want to keep it open and available for historical purposes.
We greatly appreciate all of you for bringing this to our awareness and collaborating with us while we worked towards a solution. @Greg-Bernhardt deserves a special shoutout as they have championed this thread and this issue internally.
I'll be marking this as the solution as this has been resolved and helps surface this reply for anyone who may be new to the thread.
Trevor | Community Moderator @ Shopify
- Was my reply helpful? Click Like to let me know!
- Was your question answered? Mark it as an Accepted Solution
- To learn more visit the Shopify Help Center or the Shopify Blog
This is an accepted solution.
Hello,
Thank you for your continued feedback on the web pixels issue. More recent replies have been sharing misinformation or misattribute the issue. For this reason, we've chosen to close this thread.
If you believe you have a new issue that isn't answered by our FAQ, then we'd encourage you to create a new thread in our Technical Q&A board with as much detail as possible.
Thank you.
Trevor | Community Moderator @ Shopify
- Was my reply helpful? Click Like to let me know!
- Was your question answered? Mark it as an Accepted Solution
- To learn more visit the Shopify Help Center or the Shopify Blog
See below - should I "Validate Fix" ?
I see a NEW thing, strings ending with .atom which appears to be a redirect.
Hey mont, I've been dealing with the .atom 404's for months now. It was never too excessive so I never complained. You aren't alone though. Seems like every new shopify feature comes with a new problem on GSC
We’re in free fall. Sad after so much work.
To: Ladies & Gentlemen Shopify Customers,
It is clear that we are not going to get the answers and full senior attention of the Shopify CEO by ONLY posting to this thread. My last post was quite clear (from several weeks ago), basically my request to Shopify was "tell me what you are doing and tell me what I need to do" ..... I was not given an answer.
So ..... I suggest that each person still post their issues on this blog AND then follow-up with a phone call into Shopify, state that your issue is "blah blah" (which you posted on this blog - BEFORE you call) and then say "I want to escalate to the CEO".
This provides a clear document trail. I suggest just stick to the issue and the impact it is having ... nothing else.
Hopefully such an approach will get Shopify's CEO attention.
It is clear - "hoping" that the CEO finds this blog isn't happening .... and why would it .... there are hundreds of these blog. So .... I suggest follow the above procedure and then followed by "pick up the phone".
Best regards & respectfully.
I was contacting their support, all they know is to let me resubmit the sitemap and they insist claim that there's all green lights from their end and no issue has been found. Just to let All of you know about my finding today.
Hello,
My website's rankings are decreasing every day, and I find it difficult to accept that my hard work is going down the drain. It feels like my website is dying due to these decreasing rankings.
Ok it’s not me - it’s you (Shopify)
I have two stores, same business, same brand.
One uses another platform in Canada and it’s trending upwards. Shopify platform in USA going downhill.
Other than location and platform trying to decipher differences. One is Google Merchant account. Could that be a contributing factor?
No need Shopify analytics = no need for WPM/WEB pages = problem solved
Dear Shopify Officials,
You are making some updates for our sites. I appreciate that very much.
But we are experiencing a lot of damage due to the last analytical or update/development you tried to do. What to do is stop the last update/development. You can improve the update you have tried by installing a few pilot sites, and you can fix the errors (by seeing what we are experiencing right now) without reflecting on us.
We see that you forget the warnings that you constantly remind us (make a copy before making changes to your theme, so that if there is a problem, we can return to the old order without any problems). Never too late for anything! Problems can always be fixed, which requires acknowledgment of the problem!
A warning to me below?
I don't understand!
I am new to this, if I removed Shopify analytics, is the non-index issue be resolved? Thanks
nice question. an easy solution.
If the diagnosis is correct! It is also unknown whether the problem is analytical, we assume so!
This issue has popped back up for us in a big way. Normal crawl requests to our Japan subdomain is about 2k-3k per day. This past Sunday 4/16 that spiked up to over 150k and they are all these exact links. Is there any update on what is going on from Shopify? Thanks. @Greg-Bernhardt
Hello.
Thank you to everyone who has shared their feedback. We're currently investigating this issue and are working on a FAQ that will help address the most common concerns. We will update you here once we have this FAQ ready.
I will be marking my reply as the solution as this is the easiest way to surface the most up to date information on the current status of this thread. I will remove the solution once the FAQ is ready.
Thank you.
Trevor | Community Moderator @ Shopify
- Was my reply helpful? Click Like to let me know!
- Was your question answered? Mark it as an Accepted Solution
- To learn more visit the Shopify Help Center or the Shopify Blog
Hi Trevor, looking forward to the update. Can we expect to see a solution, or should we expect to only see an explanation of the events that transpired? The words "FAQ" and "help address" aren't inspiring confidence that this is going to be an issue that Shopify fixes on Shopify's side using Shopify's time and resources.
So far I've seen a lot of blame on Google which is weird because our traffic dropped before the core update. Our first major traffic drop was the 13th into the 14th. The google core update rollout was officially started between the 15th and the 22nd, finishing on the 28th.
I know you guys are doing what you can, but the sooner we get guidance the better. I have a client who has pretty much demanded I submit almost 50,000 301 redirects for them today, one for every page and every single version of WPM that was released, they're desperate and getting ready to furlough people due to the reduced traffic. I'm having to play defense for Shopify here and so far it has not been easy.
I've watched this client make a ton of sales on Shopify. They're one of my top clients as well. Lets get this fixed so we both don't lose an excellent customer.
Hi Kbarcant, can we connect? We both kind of having a similar situation and our google search console graph basically show the same pattern. I wish we can both sort this out. I will private message you in a short while. 🙂
We have same situation with drop but without "not indexed" increase(enormous "not indexed" was bit earlier) My guess because google was ignoring all params url in general. Right now they change logic and somehow make intact in SEO.
If everyone can please DM me a screenshot of your GSC noindex report along with the impression line as @DariusWS and @kbarcant did above that would be a great help, thanks!
To learn more visit the Shopify Help Center or the Community Blog.
Wow, things have really changed since I last looked at this! On 4-18-23 I was a t 2.37M not indexed 21.6K indexed. Today, 4-26-23, I'm at 961K indexed and 6.76 indexed. I haven't seen these low numbers since before the exploits began for me in late November around Thanksgiving.
I do still have the following code in robots.txt but no removals in GSC:
<!-- Begin code to prevent spam URL exploiting search function -->
{%- if request.page_type == 'search' and search.performed and search.results_count == 0 -%} <meta name="robots" content="noindex,nofollow" /> {%- endif -%}
<!-- End code to prevent spam URL exploiting search function -->
Hi,
I don't know how to DM you, but here's an overview of one of my sites.
Besides the enormous increase of the "web-pixels" and "wpm" URLs found in GSC, my index gets bloated by a large number of dynamic URLs coming out of "Product recommendation" switched on. Google sorted these out in the past, now these days Google started to index them and created many indexed content duplicates. I have to switch this functionality off as it does more harm than good.
We had the same problem longer ago. The URL strcuture was:
/recommendations/products?product_id=6963005XXXXXX&limit=4§ion_id=template--16191536XXXXXX__product-recommendations
As far as i remember, those URLs were not indexed, they were only crawled. Anyway, it is better to exclude these from crawling and to optimize crawling budget. To solve this, you need to add a robots.txt statement.
In our case it was neccesary to add the following robots.txt statement:
Disallow: /*section_id=template--*
This was part of the URL with the product-recommendations. There are also different other robots.txt statements you could exclude those links with, e.g.:
Disallow: /recommendations/*product-recommendations*
To test, if the statement is working, you can use the robots.txt Testing Tool: https://support.google.com/webmasters/answer/6062598?hl=en
2 Things to concider:
We are continuing to see a rise in “not indexed” pages with every crawl.
404s look to have peaked in early April however, we keep seeing an increase in “Excluded by ‘noindex’ tag” and “Crawled - currently not indexed” in GSC. Of note, the last crawl saw a big spike in “Crawled - currently not indexed” for the wpm@ pages.
We can also confirm the same pattern many store owners are reporting of a sharp decline in search impressions reported by GSC starting in mid-March.
Wait for the document. Do not use the temporary URL removal tool in the mean time, it does not have an undo function.
Hi @romko18
Can you give us more information on the large number of dynamic urls coming out of "product recommedation" . I havent noticed anything like that and if that is effecting my GSC indexing i would have to turn it off too!
IMO Shopify nor Google will solve this… no worries, their prerogative, but we can’t wait any longer.
Yes, we can't wait longer, my store has a huge impact on this. I simply just did nothing and it just suddenly hits me.
@Monteveryone is at the end of their rope. I've pretty much held off my clients for the weekend but I'm seeing big changes coming in my life if Shopify doesn't fix this or get Google to intervene somehow. Unfortunately some of the smaller stores I talk to will likely be better off with new domains, the small traffic they were getting is now near zero so why not? Why build on a foundation of trash? This whole situation makes me sick. It's a shame Gilbert Gottfried is gone, I would've paid to have him read some of the customer service we've gotten, maybe make this situation a little more light-hearted.
Google is still discovering new WPM pages on their site too, shopify isn't even doing a good job hiding them. On 4/23 google crawled 17 new wpm@0.0.290 pages for one of my clients. Google shouldn't be wasting our crawl budget on these. The crawler shouldn't be able to see them at all.
About the Google Temporary Removal Tool
Unfortunately I've gotten some messages from people about to use this tool not understanding what it does. Do not use this tool unless you are 100% sure you understand how it works, what it does, and what it can do to your website. In my opinion, there's a non-zero chance of apocalyptic results. This situation also sounds like it firmly belongs under the "When Not To Use This Tool" category on their documentation Source.
Why The Temporary Removal Tool Might Be An Awful Idea
In my opinion this tool should not be utilized as a blanket solution.
Hi @shadi1 ,
the dynamic product recommendation URLs contain the same pattern: "prod_strat".
In the past, they were "Discovered, but not indexed". Some weeks, or months ago I've seen that Google started to consider them more, I found some in 404, and many were in the "Discovered" section. This was telling me I have to exclude crawling them in Robots.txt.
But this didn't obviously help to prevent indexing them. To set "noindex" in Shopify is not so easy and in my case, it didn't help:
Thanks @romko18 Another issue to look out for. I have been noticing false reporting or phantom sessions on shopify live traffic reporting. Its reporting alot more traffic then what google analytic live view is. Not sure whats going on but maybe its related to what shopify team is doing to fix this issue?
Any one experiencing this on there shopify live traffic reporting?
@shadi1 I had 88 people from Singapore yesterday before lunch, for a second I thought my traffic was picking back up.
I've had nearly 400 visits from Singapore today! It get's weirder and weirder this topic!
Hi @DariusWS I do not know how to DM you but I can not wait any longer. I am doing optimization and just when I started growing it heated me. And I see no solution. I have hired an agency they repeatedly crawl but it doe not help. I am short before using robots.
I am still waiting for the solution. Can we have an estimated rollout date for the outcome?
Quick update to not let you all hang in suspense over the weekend. Multiple teams have gone to great lengths and coordinated a proposed change in how we are delivering the web pixels that should address many of the concerns communicated in this thread. We will be testing the change next week.
When this change is rolled out there will be an official communication by Shopify. Thank you all for your patience, cooperation with me and other support staff, and for looking after each other. Have a great weekend!
To learn more visit the Shopify Help Center or the Community Blog.
Thank you for the update Greg, A lot of us are experiencing a huge increase on our shopify website visitors report , its not showing on google. I take it the testing has started and hopefully things will get better by next week!
Hopefully, Shopify give us an option of using their analytics or turning it off along with the pixels - in reality Shopify analytics are of no use to me
Shopify @Greg-Bernhardt I'm still getting new 404s - fifteen new ones over the weekend. I'm tired of watching my clients - and the entire community - lose years worth of SEO progress.
If the fix isn't coming today we should told clearly and concisely when to expect it.
Hi Everyone,
We did another audit in GSC today to check-in on the level of non-indexed pages. They are still increasing. However, after the last crawl we noticed what appears to be new attempt by spammers to inject content. This time they are using the collections/all pathway.
For now, the pages are showing up as “Excluded by ‘noindex’ tag” and “Crawled - currently not indexed” in GSC
Are other store owners starting to see this? Thankfully, the pages are not getting indexed but this could be yet another SPAM signal for Google that could affect ranking. ☹ For us, this started around April 26th
Screenshot attached…
Maybe this needs its own thread?
Thanks
Same issue for me, starting to accumulate this trash in exclude by no-index - maybe coincidence (but I do not think so) my keyword positioning in Google has dropped 30% along with my traffic since this pixel issue started.
When will we get a response from Shopify??
I have the same problem with the Spam-URLs in GSC - those links are appearing in our GSC too. None of these will be indexed - probably because those people didn't understand that the indexing of those pages have been fixed.
To avoid the flood of those URLs in GSC, i guess the only thing that can be done is to make those pages not available by 404 or 301. In my case, those are links with /collections/*collectionname*/*something-random*.
Maybe it is possible to redirect from *something-random* to *collection-name* or to 404 links with the *something-random*-URL Structure.
Despite setting the "Spam URLs" - i.e. URLs containing "wpm@" to "no index", in my case I see them indexed in GSC. I didn't do a full check, but randomly I found around 70 of them indexed.
Gentlemen, developers, due to the fact that you cannot solve this problem for six months, site owners on your platform are experiencing problems with Google. A huge number of non-indexed pages negatively affects the SEO of sites, respectively, we incur losses from a decrease in SEO traffic. But at the same time, VVSHV company regularly writes off the subscription fee from us. Question: for what? Platform malfunctions have led us to lost profits due to your slowness. We are forced to hire specialists to resolve the issue with indexing. But nothing changes. I demand compensation for the poor service of your platform. We pay money to make everything work, but in fact shopify led us to losses. Here is an example of three of my sites.
Google is off our plate now, up to last fall, it accounted for 85% of our exposure. The only traffic now is from prior customers or referrals. Needless to say not good.
@Montthe worst part is even if we move our domains to one of the countless other alternatives we're stuck with this SEO hit. The damage wasn't caused to the website that shopify hosts, the damage was caused to the URL the shopify store owner owns. I have two customers on standby waiting for me to pull the trigger and change their domains - one already bought their new URL. I'm done telling my customers "trust me, they've got this, just give it a few more days", and I'm shocked that no news outlets have reported on this yet. 30-60% traffic drop across the board because of an 'oopsie' made by the dev team. The "Worst SEO Nightmares" SEMRush 5 hr live event from 2020 covers this topic in depth and explains how to avoid the 'indexing loop' we got stuck in. Shopify knows exactly where they went wrong, and they knew how to fix it correctly before they got into the indexing loop. One can only hope this comes up during their q1 earnings call today.
Agreed...damage is done and not just domain also Brand. Yup investor relations will have work ahead to explain this.
Another crawl and more collections/all SPAM URLs appearing around +/- April 30th. (sample links attached)
Spammy links are now also appearing in the "Duplicate without user-selected canonical" category in GSC
Thanks for flagging new ones, as this gives us a chance to no index any such pages in advance. I'd implore everyone does this.
Also getting an inordinate amount of 404 redirects due to this issue. When is this getting resolved Shopify?
Hi @IanSEM, what do you mean by a 404 redirect?
To learn more visit the Shopify Help Center or the Community Blog.
Hey Community! As the holiday season unfolds, we want to extend heartfelt thanks to a...
By JasonH Dec 6, 2024Dropshipping, a high-growth, $226 billion-dollar industry, remains a highly dynamic bus...
By JasonH Nov 27, 2024Hey Community! It’s time to share some appreciation and celebrate what we have accomplis...
By JasonH Nov 14, 2024