Discuss and resolve questions on Liquid, JavaScript, themes, sales channels, and site speed enhancements.
Shopify has a new bug - they are indexing tons of useless pages in Google again. Is there any way to stop this from happening? The code seems to only be editable by shopify since it's in the content_for_header.
Do a google search for:
inurl:/web-pixels-manager@
(https://www.google.com/search?q=inurl%3A%2Fweb-pixels-manager%40)
Each site has multiple versions of this page indexed. Peets.com for example has 20.
The pages look blank, but include something like this in the code:
<!DOCTYPE html>
<html>
<head>
<meta charset="utf-8">
<title>Web Pixels Manager Sandbox</title>
<script id="web-pixels-manager-setup">(function e(e,n,a,o,t){e&&(window.Shopify=window.Shopify||{},window.Shopify.analytics=window.Shopify.analytics||{},window.Shopify.analytics.replayQueue=[],window.Shopify.analytics.publish=function(e,n,a){window.Shopify.analytics.replayQueue.push([e,n,a])});var r,i,s,l,d,c,p,u,f=a+"/"+o+"."+function(){var e="legacy",n="unknown",a=null,o=navigator.userAgent.match(/(Firefox|Chrome)\/(\d+)/i),t=navigator.userAgent.match(/(Edg)\/(\d+)/i),r=navigator.userAgent.match(/(Version)\/(\d+)(.+)(Safari)\/(\d+)/i);r?(n="safari",a=parseInt(r[2],10)):t?(n="edge",a=parseInt(t[2],10)):o&&(n=o[1].toLocaleLowerCase(),a=parseInt(o[2],10));var i={chrome:60,firefox:55,safari:11,edge:80}[n];return void 0!==i&&null!==a&&i<=a&&(e="modern"),e}()+".js";r={src:f,async:!0,onload:function(){if(e){var a=window.webPixelsManager.init(e);n(a),window.Shopify.analytics.replayQueue.forEach((function(e){a.publishCustomEvent(e[0],e[1],e[2])})),window.Shopify.analytics.replayQueue=[],window.Shopify.analytics.publish=a.publishCustomEvent}},onerror:function(){var n=(e.storefrontBaseUrl?e.storefrontBaseUrl.replace(/\/$/,""):self.location.origin)+"/.well-known/shopify/monorail/unstable/produce_batch",a=JSON.stringify({metadata:{event_sent_at_ms:(new Date).getTime()},events:[{schema_id:"web_pixels_manager_load/2.0",payload:{version:t||"latest",page_url:self.location.href,status:"failed",error_msg:f+" has failed to load"},metadata:{event_created_at_ms:(new Date).getTime()}}]});try{if(self.navigator.sendBeacon.bind(self.navigator)(n,a))return!0}catch(e){}const o=new XMLHttpRequest;try{return o.open("POST",n,!0),o.setRequestHeader("Content-Type","text/plain"),o.send(a),!0}catch(e){console&&console.warn&&console.warn("[Web Pixels Manager] Got an unhandled error while logging a load error.")}return!1}},i=document.createElement("script"),s=r.src,l=r.async||!0,d=r.onload,c=r.onerror,p=document.head,u=document.body,i.async=l,i.src=s,d&&i.addEventListener("load",d),c&&i.addEventListener("error",c),p?p.appendChild(i):u?u.appendChild(i):console.error("Did not find a head or body element to append the script")})(null,null,"https://cdn.shopify.com/shopifycloud/web-pixels-manager/0.0.186","sandbox","0.0.186");</script>
</head>
<body></body>
</html>
Solved! Go to the solution
This is an accepted solution.
Hello everyone.
Thank you for your patience while we worked towards providing a fix to this issue and building out an FAQ that addresses the primary concerns and questions from this thread. You can find our FAQ on the Community Blog.
We welcome you to continue the conversation in the blog section of the blog post with any valid questions that aren't already answered by the FAQ. We will monitor the comments section for any valid feedback on how this change may impact Shopify stores and will actively remove or edit any comments that spread misinformation or speculation on the issue.
On that, this thread contains some misinformation and speculation on the issue and how it may impact ones online store. For this reason, we will no longer be monitoring this thread, but want to keep it open and available for historical purposes.
We greatly appreciate all of you for bringing this to our awareness and collaborating with us while we worked towards a solution. @Greg-Bernhardt deserves a special shoutout as they have championed this thread and this issue internally.
I'll be marking this as the solution as this has been resolved and helps surface this reply for anyone who may be new to the thread.
Trevor | Community Moderator @ Shopify
- Was my reply helpful? Click Like to let me know!
- Was your question answered? Mark it as an Accepted Solution
- To learn more visit the Shopify Help Center or the Shopify Blog
This is an accepted solution.
Hello,
Thank you for your continued feedback on the web pixels issue. More recent replies have been sharing misinformation or misattribute the issue. For this reason, we've chosen to close this thread.
If you believe you have a new issue that isn't answered by our FAQ, then we'd encourage you to create a new thread in our Technical Q&A board with as much detail as possible.
Thank you.
Trevor | Community Moderator @ Shopify
- Was my reply helpful? Click Like to let me know!
- Was your question answered? Mark it as an Accepted Solution
- To learn more visit the Shopify Help Center or the Shopify Blog
@ahsonmkhan the first linked URL is noindex and the second is 404
To learn more visit the Shopify Help Center or the Community Blog.
@Greg-Bernhardt Thank you for all the support you have extended. We hope these will be sorted out soon.
Hi Greg, there is no noindex in wpm@ pages and our site produces such pages everyday. Google still indexed them even we have add Disallow:/wpm@* in the robots.txt.
Our site had indexed about 3 hundreds pages before. Now google shows more than 6000 pages and our traffic dropped by 2/3. Can you guys stop the code to produce such no content, meaningless pages?
@Greg-Bernhardt Can you guys stop the code to produce such no content, meaningless pages?
In one night one of my sites went from under 100 freshly fixed 404's to over 3,000. The next day (3/15) I'm down to 75% of the traffic 3/16 I'm at 50%. It's stayed steady since.
We need to get this resolved ASAP, this is not going to be easy to explain to the business owner why half of his traffic vanished. Please, any update would be appreciated. I'll be tapping F5.
its really bad that shopify is completely not addressing this issue when they are the CMS, majority of stores on teh platform are being affected.
I went to do a check on the sites in my country, a ton of them even big names are affected
Hi
Where did the /web-pixel-manager@ come from, i have the /wpm@ issue going on right now. I had the web pixel manager issue and it was supposedly fixed few month ago.
The /wpm@ is getting fixed right now, my indexed pages dropped by 1000 last night, i tested to make sure @WPM is not blocked by robot.txt any more and its also not being indexed .
So The /wpm@ pages are dropping . Your screen shots are scaring me thou , I m really hoping the /web-pixels manager@ is not back now.
My Sales traffic and sales have also had a big drop the past few weeks. I haven't heard anything about compensation yet . This is not our fault we should be compensated some how!!!
Yeah, we need compensation !!!!!
Please someone tell me if this makes sense , because it appears they're playing whack a mole my little store can't take being slammed by Google, they won't run my ads even. Am I correct the issue is that our auto generated sitemap.xml includes these bad urls. What if built my own tidy sitemap and sent that to Google instead.
OK Interesting...just went to collect my site map and "Products" was changed by shopify from https://vr.supplies/sitemap_products_1.xml.
to
https://vr.supplies/sitemap_products_1.xml?from=1680358867017&to=6815972884560
In essence removing bad urls (404) that were there yesterday from reaching Google. I just sent a new index request to Google Console. Fingers crossed it worked.
Just went to collect my site map and "Products" was changed by shopify from https://vr.supplies/sitemap_products_1.xml.
to
https://vr.supplies/sitemap_products_1.xml?from=1680358867017&to=6815972884560
In essence removing bad urls (404) that were there friday from reaching Google. I just sent a new index request to Google Console. Fingers crossed it worked.
Are they seriously not going to resolve this? The silence is deafening.
Both of these things will take weeks to complete. You will see an increase in your "pages with noindex" report and that is a nonissue in terms of SEO.
To learn more visit the Shopify Help Center or the Community Blog.
What is the shopify's method? Add noindex tag in wpm pages? I did not find such tag in these pages at present.
You said nonissue in terms of SEO. Now Google has indexed these pages. If your guys do not know what is spam from Google, here is the definition from Google https://developers.google.com/search/docs/essentials/spam-policies .
Spammy automatically generated (or "auto-generated") content is content that's been generated programmatically without producing anything original or adding sufficient value; instead, it's been generated for the primary purpose of manipulating search rankings and not helping users. Examples of spammy auto-generated content include:
If you're hosting such content on your site, you can use these methods to exclude them from Search.
@jackzhu the noindex is sent via http header, not meta tag
To learn more visit the Shopify Help Center or the Community Blog.
@Greg-Bernhardt i see it. But our site also still get such pages indexed. Why shopify could not stop the code to produce such pages?
"pages with noindex" are not problematic, if they are out of the index. The only thing that might need to be monitored is the crawl budget after everything is fixed.
The problem we had/have is "indexed, although blocked by robots.txt".
The GSC is usually also lagging a few days behind. That's why it might look like pages are still going to be indexed, which is not the case.
As long as the robots.txt doesn't block those pages (you can test the affected URLs here) & noindex-Tag is added to those sites (X-Robots or Meta robots, you can test the affected URLs here here), no more pages are going to be indexed.
The only thing you can do to clear the index faster is to go to "indexed, although blocked by robots.txt" in Search Console and start a review. You'll probably need several runs to get them all out of the index.
"web-pixel-manager URLs will 404 and drop from the index"
How? Is something changing in the future? Right now /web-pixels-manager pages are not blocked in the sitemap, not blocked with no-index tags, and not blocked with x-robots tags. What has changed and how can we verify that it's actually changed?
I can't tell google the problem is fixed and have them spend an exorbitant amount of our crawl budget over the next month to see that the pages are still indexable and being served by shopify. Stuff happens. Just be transparent with us and tell us what the roadmap is because saying indexable pages "will 404 and drop from the index" is a promise that seems unrealistic.
@kbarcant can you please DM that link, thanks!
To learn more visit the Shopify Help Center or the Community Blog.
Dm'd.
@Greg-Bernhardt -This issue has draggged on for nearly 3 months. For clarification, can you confirm:
1 - You state that the sandbox url's are "no-index" in the http header - but when I check one of my product skus url with the sandbox url
https://www.bunkbedking.com/[email protected]/sandbox/products/bunk-beds-tall-stairway-gray...
(see screenshot) it shows that google can index:
https://www.dropbox.com/s/v1gjkjr6wrdunrw/Screenshot%202023-03-26%2010.49.19.png?dl=0
I think part of the problem is that shopify has created many url options of this sandbox pixel and now I have them scattered around in Google Page Indexing - I have versions in:
Blocked by Robots.txt
Crawled - currently not indexed
Excluded by ‘noindex’ tag
Not found (404)
So can you confirm exactly what steps are needed to clean up this mess for each of these 4 categories?
Reading thru your comments, it seems the solution is to do nothing and let the shopify created url fix the issue. If so, you need to go back to all the versions of this sandbox url shopify created and make them not not indexable ie as the [email protected]@ variant
https://www.dropbox.com/s/83dlit8nj3h96wi/Screenshot%202023-03-26%2011.08.55.png?dl=0
From our side;
We should not add a no-index, no-follow tag in Robots.txt for the sandbox url's?
We should not disavow the sandbox url's?
Thanks
Denis
as an aside why was this code not tested before it went live to ensure it functioned correctly?
I'm astounded at the patience of this community. Clearly a lot of knowledgeable people in here who understand the impact of this issue. I don't have anything to contribute except to add that I have the same issue and have regularly been frustrated with Shopify's inability to address obvious issues in a timely manner (setting accurate times for shipping zones being one - still unresolved for the UK). It is clear that smaller retailers are all but ignored on this platform.
Hi @Greg-Bernhardt when can we expect shopify to come out with a fix for this...?
Just an estimated date of when the devs can fix this issue that is plaguing many merchants
@Jon can you please articulate the exact issue you are facing?
To learn more visit the Shopify Help Center or the Community Blog.
@Denis_B the pixel URL you provided redirects to https://www.bunkbedking.com/collections/bunk-beds-stairway. You'll want to investigate why.
@tom511 what report is that screenshot from? The new URL is from search, not web pixel.
To learn more visit the Shopify Help Center or the Community Blog.
The rest of the indexed pages are getting deindexed slowly however I have seen a new indexed page today. Please guide me if there is any index tag that needs to be added like vendors/collections.
Thanks for the link @ahsonmkhan, I'll take this back to the devs
To learn more visit the Shopify Help Center or the Community Blog.
Also please check if we can use this solution for the time being (Read this on another community post)
{% if request.path == '/search' and search.results_count == 0 %}
{% endif %}
@ahsonmkhan yes you can use that. Once implemented, please confirm it's working.
To learn more visit the Shopify Help Center or the Community Blog.
@ahsonmkhan that page is sending noindex via header but your theme is injecting a robots meta which says to index.
To learn more visit the Shopify Help Center or the Community Blog.
@Greg-Bernhardt This is a bit complicated for me. please suggest should I go ahead with putting that noindex tag in the head section only
or is there anything else that should be done along with it
We all have lots of wpm@ urls in GSC indexed page. @Greg-Bernhardt Now what is the solution for these pages?
Hi Greg
Response please to my other questions:
I think part of the problem is that shopify has created many url options of this sandbox pixel and now I have them scattered around in Google Page Indexing - I have versions in:
Blocked by Robots.txt
Crawled - currently not indexed
Excluded by ‘noindex’ tag
Not found (404)
So can you confirm exactly what steps are needed to clean up this mess for each of these 4 categories?
Reading thru your comments, it seems the solution is to do nothing and let the shopify created url fix the issue. If so, you need to go back to all the versions of this sandbox url shopify created and make them not not indexable ie as the [email protected]@ variant
https://www.dropbox.com/s/83dlit8nj3h96wi/Screenshot%202023-03-26%2011.08.55.png?dl=0
From our side;
We should not add a no-index, no-follow tag in Robots.txt for the sandbox url's?
We should not disavow the sandbox url's?
Thanks
Denis
as an aside why was this code not tested before it went live to ensure it functioned correctly?
Blocked by Robots.txt: These will reduce over time as we removed the block
Crawled - currently not indexed: this report is not an issue in this instance
Excluded by ‘noindex’ tag: this report is not an issue in this instance
Not found (404): this report is not an issue in this instance
To learn more visit the Shopify Help Center or the Community Blog.
The problem is still not solved knowing that it is more than 2 weeks old for the number 1 host of ecommerce site it becomes shameful. Each day more is less credibility for all your users. Moreover, even today I still have the Wpm and Web pixel Manager pages that appear but I also have a Chinese URL that was not created by me, which has just been detected. So I'm going to ask users if this is related to the same problem as WPM and if they too have detected this because shopify doesn't seem to want to help us this PROBLEM, or doesn't communicate enough which makes everything unclear and very unpleasant.
Why are you doing this on Earth? Do you know how hard we effort on our webs? While you can just leave with a tiny code bug bomb for us to deal with the shitty results? We don't accept it! YOU SOLVE THIS PROBLEM AT ONCE, AND YOU OWE US AN APOLOGY!
@Greg-Bernhardt We are also experiencing this issue on our store. Should I DM you with details?
Organic search result from Analytics for March 2023 if down by 36% from March 2022.
Can you provide an update on this? I agree with @TIMECAPSULES , it is not clear from these 9 pages what we have to do to fix this issue. You keep asking for individual store pages with issues but what is the generic fix every store owner can use to resolve the indexing problem?
Can you please provide an update? Or at least inform us on the status of fixing this? My SEO ranking is dropping severely, similar to other's. Please communicate a clear path forward or at least provide some insights from Shopify's side into the timeline of fixing this. Doing so also removes the urge for other's to keep contacting you about the same matter.
Thanks in advance.
This is definitely not fixed. We have clients sending us their GSC error notification emails on a weekly basis. Now the error is showing up as a Server error (5xx) in GSC.
https://website.com/[email protected]@5c46ad45w03c6b27bp74bf52d0m9e637fdc/web-pixel-shopify-custom-pixel@0530/sandbox/collections/collection-name
I have a question for all - just curious. Of the few - throughout this community with similar issues going back for some as early as last summer - I noticed sites from EU, Canada etc... also NOT dot.com's
SO Questions are: What country is your affected site in? What is affected site domain name end suffix?
Just wondering if there's a pattern re outside the USA or Not a dot com
Great question. My issue is strictly on our Japan subdomain right now
Hi,
I've been following this issue. Our site's ranking has dropped by almost 40% since March. According to the latest Google report, although there are not generating any new indexed pages (web or wpm), there are still constantly generating new web 404 pages and noindexed wpm. Theoretically, these pages shouldn't impact our rankings. However, the true problem is that our rankings aren't showing any signs of improvement and are continuing to worsen. I'm unsure about what steps to take next, and I'm feeling very anxious about the situation.
Is there a list of all text affecting urls? This is what I've flagged and no indexed so far. Be good to add in any other text no so google doesn't index my pages if I'm affected by them.
collections/vendors?q=
[email protected]/sandbox/
[email protected]@1543f512w2542bf6cpa9acea22m3c2685ed/sandbox/blogs/news
[email protected]@a1fcb853wff0d51fapfe890cd8ma6042ba1/sandbox/
Hello,
Given the current large number of noindex wpm pages, would disallowing them be more beneficial to reduce Google's crawling costs?
Thanks!
Omg, happy to see people are having same issues as me. My traffic is turning BIG downwards due to this issue since Middle of March core update. Tons of pages are not indexed currently. Can anyone suggest me solution?
So far shopify solution has been to no index, they placed a no index tag in the headers of all stores, they can't use a disallow tag in robot.txt as that would create a lot of indexed thou blocked by robot.txt pages.
I thought this would of been a temp solution and i m trying to find out if there a permanent one in the works!
Any news on that @Greg-Bernhardt
The problem is that this issue has clogged up my no-index list in GSC - so i cannot see if any of my real skus are not being indexed - frustrating
I would really like to see a response from Shopify of their plan to fix the current issue and ensure it is not an ongoing problem (not just comments from Greg who I thik is trying to do his best, but has no complete answers that solve our concerns)
Agree, would appreciate knowing what Shopify intends to do beyond temp fixes, if anything.... or maybe they can't or don't know. Greg thank you for doing all you can.
I suspect Google's A.I. and algorithms will fix their problem with us. Who knows where that will lead. If we go one more month with SEO pointing in the wrong direction...whatever the cause. It just isn't sustainable for our Shopify store.
Starting a B2B store is a big undertaking that requires careful planning and execution. W...
By JasonH Sep 23, 2024By investing 30 minutes of your time, you can unlock the potential for increased sales,...
By Jacqui Sep 11, 2024We appreciate the diverse ways you participate in and engage with the Shopify Communi...
By JasonH Sep 9, 2024