Have your say in Community Polls: What was/is your greatest motivation to start your own business?

Shopify - Web Pixels Manager Sandbox FAQ

Trevor
Shopify Staff (Retired)

Summary of Events

Beginning in January of 2023, some merchants reported seeing a large amount of web pixel URLs being indexed in their Google Search Console. Our team reviewed the issue and searched for “web pixel manager sandbox” in Google. We confirmed that the URL of the iframe sandbox was being indexed and started working on a resolution. As a result, we partially mitigated the issue by 404-ing the old web-pixel-manager paths that were previously indexed, preventing the new path from being indexed. 

 

Solution Moving Forward

We have created a solution to stop the crawlers from executing the JavaScript preventing pixel URLs from being crawled. This solution will help solve the issues our merchants are facing as outlined below:

 

  • New pixel URLs should not show up in merchant’s GSC reports
  • Merchant Google Search Console noindex reports should clear after a few months, (we are unable to offer a specific timeframe) as no new pixel URLs should be added. 
  • Merchant Google Search Console 404 reports will temporarily increase as the old pixel URLs that were noindexed will transition to 410s. After several months (we are unable to offer a specific timeframe), the 404 report should also start clearing.

 

Merchant SEO Concerns and FAQ

 

Do the 404’d and noindexed pixel URLs affect search engine crawling and crawl budget?

Google has confirmed to us that these 404 and noindex pages will not affect your crawl budget or Google’s crawl effectiveness of your store.

 

How long does it take for the old pixel URLs to drop from the Google index?

It depends on how many old pixel pages were indexed and the frequency of Google’s crawling of your store. For a site with many old indexed pixel pages with low frequency crawl rate, it could take months for these pages to drop from the index.

 

Is Shopify still linking to the old URLs that are now 404ing?

No, the old /web-pixels-manager/ URLs are no longer being used and are not linked anywhere from your site. These pages are now orphaned URLs that no longer exist anywhere.

 

Should you redirect all the 404s to other pages?

No, these old pixel pages no longer exist and were of no value to Google. These URLs should remain 404’d. See Google’s official documentation on 404s.

 

Do the old 404’d pixel URLs affect search engine rank and traffic performance?

No, the 404s will not hurt your SEO performance. The pixel pages are of no value to search engines and they are not linked in any way from your store website. See Google’s official documentation on 404s.

 

Do the current noindexed pixel URLs affect search engine rank and traffic performance?

No. See Google’s official documentation on noindex and comments from representatives here.

 

Should you add a robots.txt rule to block crawling of the pixel URLs?

No, blocking the URLs via a robots.txt disallow rule may prevent search engines from seeing the noindex header for the pixels pages, along with flooding your “content blocked via robots.txt” GSC report.

 

Is there a way to clear the 404 and noindex reports in GSC?

At the moment there is no way to clear these URLs from your reports. A workaround is choosing “Select all submitted pages” from the top left dropdown menu when viewing these GSC reports instead of “All known pages”. That will filter the reports to only use URLs in your sitemap.

 

Should you change domains as a method of clearing the GSC reports?

No, migrating your store to a new domain is strongly not recommended since it could cause great disruption in your store’s SEO performance.

 

Why has my Google SEO changed/faltered recently, in general?

With the understanding that the 404ing and noindexed URLs as confirmed by Google are not harming SEO performance, we would suggest reading up on the 6 major Google algorithm updates released since the pixel URLs were introduced. Following this link you can learn more about each update, how it may be affecting your store, and what you can do to improve SEO performance.

67 Comments
kbarcant
Shopify Partner

So my 50% drop in traffic two days before the Google update was because of the Google update??? 

jasonwill21
Excursionist

Hope it helps with bringing back our SEO traffic.

AngryLuisatwpb
Tourist

Yes that help a lot, and it solved my problems, crawling problem was solved indeed, that's never a problem. And this blog also helps my keywords ranking goes up. Thank you guys for what you done.

Denis_B
Excursionist

"We have created a solution to stop the crawlers from executing the JavaScript preventing pixel URLs from being crawled"

 

What is the solution?

Denis_B
Excursionist

So the action we should take is to do nothing and wait for all the pixel urls to disapear from the google reports?

 

I went ahead and temporarily removed in Google the pixel urls that were indexed by Google. Should I request cancelation or let them expire?

DariusWS
Shopify Partner

This fix will fix recommendation bug issue too?  

DariusWS_0-1684222029531.png

 

romko18
Tourist

@DariusWS - I'm also having this problem with product recommendation URLs, but this thread is not about that. I'm already working on a workaround for this.

shadi1
Explorer

My indexed count thou blocked by robot.txt has increased by 300 all the sudden just few days ago?  I thought the no index directive was suppose to prevent that. Is the no index still working in the header?

So now how long will it be before those indexed wpm are gone and aren't they effecting my seo negatively.

Would using the removal too in GSC help?

 

 

 

p1Commerce
Shopify Partner

@Denis_B The solution works like this:
shopify-pixel-implementation.png

We personally recommend leaving the temporary removal to at least keep these pixel URLs out of the index while GSC stabilizes.

Details at: https://p1commerce.com/resources/shopify-web-pixels/

znnadmin
Shopify Partner

Hello @romko18 , I am having the same problem as you and @DariusWS ( Page indexing > Crawled - currently not indexed ) 

my-webiste.com/products/some-page?pr_prod_strat=collection_fallback&pr_rec_id=2f8bf9abf&pr_rec_pid=498839978014&pr_ref_pid=3593135425&pr_seq=uniform

Can you please share the thread so that we can share the fix if possible?

 

 

ItsMeBirdie
Visitor

So its seems that I am having this issue as well, although as of today it does not seem to have been fixed yet is there something I need to do on my end?

p1Commerce
Shopify Partner

 @romko18  @DariusWS 

 

This change won't impact the product recommendations.

WHY THESE ARE PRESENT

Shopify uses the pr_ query string variables appended to the urls for analytics to determine the success of the product recommendations in the "You may also like" section of the product pages. The tracking helps improve the recommendations over time, and I believe this is standard implementation on the Dawn theme.

IN GOOGLE SEARCH CONSOLE

Having these pages in Alternate page with proper canonical tag is good.

Having these pages in Crawled - currently not indexed is not ideal, but as long as your site has proper canonicals and these pages are indexed without the query variables, it isn't really a concern.

Web pages will almost always have instances where they get indexed with and without query strings but as long as the canonicals are correct, should be fine.

REMOVING FROM TEMPLATE

If you wanted to remove them (keeping in mind dynamic recommendations would not improve) you can remove query strings by modifying your card-product.liquid file.

FIND INSTANCES OF:

 

{{ card_product.url }}

 


AND CHANGE TO:

 

{{ card_product.url | split: "?" | first }}

 


This will remove all query strings on product urls passed to the product grid display.

SheenaSchleichr
Visitor

First off, it's unlikely that these web-pixel-manager or wpm@ URLs caused a decline in organic traffic/rankings. It was unfortunately timing that this happened around the time several G algo adjustments rolled out, but we manage a number of Shopify+ sites that were affected by this bug and none experienced a performance drop.

 

We were able to have all of the WPM URLs (both versions) de-indexed within a few days of seeing the jump in indexed pages by using Removal tool within Google Search Console. There's no reason to be scared of this tool--just use with care. 

 

You can do a site operator lookup to see which version of the WPM URLs your site has indexed by plugging this into Google: site:yourdomain.com/web-pixel-manager

 

Obviously replace "yourdomain" with your actual site's domain name. Hit enter & see what comes up. Then do the same with site:yourdomain.com/wpm@

 

As an example, here's the lookup with a wildcard (*) for the domain: 535k results!

SheenaSchleichr_2-1684360656123.png

 

Whichever version you see indexed for your domain is what should be input to the Removal tool. If it's both versions, then submit two separate removal requests with "Remove all URLs with this prefix" selected: 

 

SheenaSchleichr_0-1684360147101.png

 

SheenaSchleichr_1-1684360372760.png

 

This will cleanup WPM URLs in the index and GSC temporarily. Once the Removal expires, the pages won't reindex because of the noindex solution the Shopify implemented. Easy peasy. 

 

 

Mont
Explorer

So over my head. I just moved my little growing store to Shopify thinking it should be fairly understandable and manageable. Shouldn’t be rocket science. That was the pitch after all. Guess I need a dev team now to figure this out. 

jasonwill21
Excursionist

@SheenaSchleichr  

In my opinion, there exists a noticeable discrepancy between larger, high-authority Shopify PLUS stores and smaller merchants such as ourselves. We are undoubtedly facing significant challenges. It seems that Google consistently penalizes small merchants and new stores like ours without providing any explicit justification. As we are aware, comprehending and adapting to Google's algorithmic changes can be quite intricate, as it operates on a highly sophisticated level. Unless one has personally encountered such penalties, it is difficult to truly grasp the implications. Additionally, I have observed that a store that features an extensive array of "links" pertaining to factors like apparel sizing and colors may experience the greatest impact. Perhaps, in the future, if Google were to penalize larger stores while leaving smaller merchants unaffected, it would offer a different perspective, warranting further observation and let's see. have a Beautiful Day!

Mont
Explorer

Prepare folks. Small business  entrepreneurs that is… You may be reduced to referrals and direct email. Google may or may not be helpful to us. Time will tell but prepare for not.

IMO next decade will not be friendly to small biz. 

EnterGallery
Visitor

You mentioned:

  • New pixel URLs should not show up in merchant’s GSC reports

and this was posted on 15th May

 

I have new web pixel URLs still coming in on GSC, the latest is 19th May?

 

When is this actually likely to stop?

Mont
Explorer

Increase in 404 pages. Google asks to validate fix. ???

 

Mont_0-1684778777474.jpeg

 

EnterGallery
Visitor

For our site, this really doesn't seem to have stopped, in fact it is getting worse.

 

I recently posted that we had several of these URL's added between 15th and 19th May. Since then Google has identified over 500 more of these in our 404's.

 

It is now becoming very difficult managing our normal 404's because we are having difficulty finding them.

 

Please, how can we make this stop?

 

In desperation I am trying the Remove Tool, to block anything with '/wpm@' and '/web-pixels-manager@' for six months, in the hope this will help.

jasonwill21
Excursionist

@EnterGallery I tried the remove tool, it seems useless. Tried every sort of tools, tactics, adjustment plus this shopify update, no improvement can be seen on my situation.Screenshot 2023-05-26 at 10.27.49 AM.png 

romko18
Tourist

Hi @Trevor ,

 

I'm also seeing new "Wpm" URLs appearing in the "Not found (404)" section in GSC every day and they seem not to contain the "noindex" tag, at least in GSC they are not marked like that. Is your solution actually working?

Italia-Straps
Explorer

Another Google crawl and an audit in GSC shows a continued increase in "no index" pages and 404s. It's all the wpm pages and the collections/all spam... same as before. Are any store owners seeing their indexing stats improve since this fix has been implemented?

 

Thanks

Josh_Uebergang
Shopify Partner

You don't want WPM being indexed so noindex is good.

 

You do not need to be concerned about such Web Pixel Manager pages in GSC showing a 404. Beyond 404, legitimate 404s are fine to have for SEO. These are pages that genuinely don't exist and if they have no other variation or you don't want them indexed, it's fine to 404 them.


It seems Shopify must remove previous old versions when a new version of WPM is released. This causes the 404.

 

I'm working on a Google Ads tracking code with all features that uses WPM (or Shopify Customer Events/Pixels). In the diagnostics of Google Ads, I've seen these URLs as they are the webpage that trigger conversions. The number after the @ symbol in the URL shows the version. Customer Events was buggy since it's release. Shopify's own example in 2022 didn't even follow their guidelines and never worked. Ever since Customer Events release, I've seen the version number in the URL change almost every 2 weeks. The format today is like: 

 

 

[domain]wpm@6d6339c0w77483ddapdc20cb53m2575a67f/web-pixel-shopify-custom-pixel@0543/sandbox/[hnadle]

 

 


Remember, you can now edit the robots.txt file in Shopify, so if a similar problem happens again, you can noindex it yourself.

jasonwill21
Excursionist

Like is freaking moving, so let's move on 🙃

skclouse
Visitor

I'm still getting this error for https://libertysafe.myshopify.com/. I received a Google alert over the weekend that I'm having indexing issues for this URL.

https://www.libertysafe.com/wpm@0.0.276@8694256cw411e5e19p0cbd854bmfb657819/web-pixel-shopify-custom...

 

skclouse_0-1687189695929.png

 

I also still have these showing that was checked by Google in late May.

skclouse_0-1687189826284.png

 

 

DariusWS
Shopify Partner

Everyone who have same issue and search drop more than ~40% send yours shop url to PM. I would like to investigate correlation between projects. We have 3 projects. 2 of them on 15th of march got improvement in search and one got search DROP. It would great get more projects with drop to better investigation

znnadmin
Shopify Partner

@DariusWSI would message you but it seems that I cannot due that due to a combination of factors that are not disclosed but only mentioned in https://community.shopify.com/c/shopify-discussions/direct-messaging-members-in-shopify-community/td...

If you manage to find out something, please share with the rest of us.

I had a spike with page views in March but since then, things are downhill.

Also, I have requested removal but I still have almost the same number of invalid URLS.

znnadmin_0-1687455453456.png

 

skclouse
Visitor

@DariusWS, how do I PM you? Our store URL is 

skclouse
Visitor

Znnadmin with a platform as large as Shopify, you think that this would be a priority for them and fixed a long time ago!

tmax20hbx
Visitor
somethinspecial
Tourist

I've been seeing these URL's for months and just did a redirect. But now my 404 validation report shows 4,794 pages not found.

Not sure what I should do here?? Just wait it out?

Thanks,

Sue

DariusWS
Shopify Partner

@Skclouse and @znnadmin

Not sure if you guys haven't turn on PM functionality or just I am banned from PM

DariusWS_1-1687769520335.png

 

DariusWS_0-1687769406019.png

 

 

kbarcant
Shopify Partner

From 7-1 to 7-3 we got 4,200 new web-pixels-manager 404s on a website with <300 products. These newly discovered 404s are listing the referring URLs as their corresponding real pages, so the error is generated by Shopify.

 

Why is Search Console telling me that between 7-1 and 7-4 that it was referred to newly discovered wpm links from good pages on the website?

 
 
znnadmin
Shopify Partner

@DariusWSI also cannot PM, please see above the Shopify procedure to see how you can achieve that.

roseybluebonnie
Visitor

Following on from  @p1Commerce  advice, I have submitted /web-pixel-manager and /wpm to GWT for temporary removal. Google is no longer indexing these pages, however, we get surges of 404 error notifications in GWT - by 2,000 at a time! Every time we get a surge in 404s our SERPS decline significantly. This has happened 3 times now and correlates directly in time with SERPS position reductions. After a few months, the 404 errors decline again and ranking positions increase.
I'm tempted to add /web-pixel-manager* and /wpm* to the disavow in the robots.txt to stop the Google bot from crawling these pages. Any advice??

kbarcant
Shopify Partner

We went from <10 404s to nearly 10,000, along with literally about 10,000 other errors. We also went from 1000 daily users to 250. We've been repeatedly told it's unrelated by Shopify and that no additional fixes are being rolled out as there's no issue detected and the thread about the issue was closed. Shopify support has left my ticket open for days now with no response and I've been told to not pursue this issue further on the forums as well.

To anyone reading this: Don't do what I did. Don't waste 200 hours over three months analyzing this data expecting this to be fixable in any meaningful capacity. The ball is entirely out of our court on this issue, we can only wait to see if they return it or if they just leave us hanging.

JMartocci
Excursionist

I am having a more significant problem with this sandbox file. I am using Clarity by MSFT and noticing daily JS errors with my customers. Here is a link to one of my customers for you to see it in action. Can this file be removed from my website? 

 

broken-website-1024x590.jpeg

 

https://myrubylips.com/wpm@abd83086w6100c671p96a62313md77ea07a/web-pixel-40567057@1/sandbox/modern/ 

 

Here is the video, js error in action: https://clarity.microsoft.com/shared/recording/2c314351-2926-49fb-b091-39c2a91c380f  

 

My website (NSFW): https://MyRubyLips.com

 

Any kind of help would be awesome.

Trestlesouth
Tourist

We are facing the same issues. Sales significantly affected and 45% less traffic to our store.

Surely Shopify should be accountable for these errors .

 

Any kind of help would be appreciated.

 

nikkore
Shopify Partner

We've just recently noticed these errors on our store. As I understand, there is no manual fix to this and should just wait for it to go away? 

nikkore_0-1691973897515.png

Any other store owner out there that got affected by this, please share your thoughts. Thanks!

 

kbarcant
Shopify Partner

Hopefully this helps. I wrote this up this morning for all of the store owners like my clients who just need to understand what's going on. I did my best to put it in plain English since Shopify is directed at those who aren't necessary professional web developers. 

 

 

The Two Flavors Of Our 404 Errors

There are two types of 404 errors that you may see on your Google Search Console report: wpm 404s and real 404s. Wpm 404s are generated by the web pixel manager issue, and include with "web-pixels-manager" or "wpm" in the link. Real 404s are caused by pages on your site that cannot be found by Google. These are the ones that you need to resolve.

 

 

Limitations of Google Search Console Report

Shopify’s advice is to filter 404 errors by submitted pages only. This will remove all the wpm 404s as they are not on the sitemap. However, this will also remove real 404s because they are not on the sitemap either, thanks to Shopify’s automatically updated sitemap. Therefore, you need to find another way to check your real 404s. When I first discovered this, I had to fix a lot of bad internal links that were hurting my SEO.

Another problem with the Google Search Console report is that it only shows you a sample of the 1,000 most recently crawled 404s, not the most recently discovered ones. This means that even if a new issue arises, it may not show up on the list if Google has recrawled other pages since then. Once you have over 1,000 404 errors, your report is officially inaccurate. For example, we have 11k 404 errors, and we only see 9k of them, which is 9% of our total errors. Those odds are not great. If you have just over 1,000 errors, your report would be more accurate.

 

 

Solutions for Different Scenarios

Depending on how many 404 errors you have, you may need different solutions to handle them.

 

If your store has under 1,000 404 errors

You can use the Google Search Console report to filter out the wpm errors and fix the real ones. To do this, go to your 404 report, filter by URL, and put “Does Not Contain” and put the @ sign. This will remove all wpm errors from the report. The remaining errors are real 404s. Fix them as necessary by setting up redirects or updating links, and then manually submit the URL to Google for reindexing it. Google will see that it is a 301 redirect and it will remove it from your report.

 

If your store has over 1,000 404 errors

You will need to pay for an SEO app that tracks 404 errors. These apps only start working from when you install them, so you will need to wait for data to start rolling in. I will not recommend any specific app, as I do not want to endorse any product or service. You can manually resubmit these URL's the have them marked as fixed. 

 

 

Why Real 404 Errors Matter

Monitoring the 404 report is very important for your SEO. If you make a typo in an internal link, this is where you would see it. But there are other problems that can arise as well. For example, if you delete a collection, most store owners who self-develop would just write a 301 redirect for the collection URL, such as /collections/old ► /collections/new.

However, nested product URLs (such as /collections/old/products/exampleproduct) may also need to be redirected, and without proper access to a 404 report you would never know. There are many other examples like this and preventing customers from seeing 404 errors is a well-established best practice for SEO.

This report is the canary in the coal mine when it comes to common errors made by the exact kind of people Shopify advertises to - those with little web development experience. I think that Shopify needs to develop an indexing management tool that allows us to see what pages are serving as 404s, and preferably a better 301 redirection option with wildcards or patterns available.

 

 

Summary

Using the ‘submitted pages’ filter results in no usable data. If you have far over 1,000 404 errors, pay for a 404-tracking app until Shopify either fixes this or launches their own 404 tracker and watch for your real 404s in there. If you have under 1,000 errors, congratulations, you get to keep your money. Filter out the wpm errors with @.

shadi1
Explorer

@kbarcantthank you for this . I wanted to ask about the venders?q= in the 404 report?

Trestlesouth
Tourist

Hi Can anyone please advise if this wpm issue could have any effect on my Facebook Pixel  and Ad's

as I have noted on my datasets (on Facebook Events Manager) a number of websites linked that are not mine. Just trying to uncover why after this issues none of my traffic is returning and wether it has any correlation.

Kind Regards

GigBros
Excursionist

Are there any updates regarding this issue?
It has been on my site for months, when I asked I wasn't given a precise or clear answer.
I thought it was my problem.
I found this forum page, I noticed that there are quite a few with this problem and no definitive solution has been found.
I'm not a developer, but a store owner.
Is there a specific page on this forum where this web pixel manager sandbox problem is talked about?
Will Shopify solve this problem?

TonyRC
Visitor

It is October 25th, over 5 months since this FAQ has been created.

 

The problem still exists.

IanSEM
Tourist

Still seeing an inordinate amount of these WPM Sandbox 404s in my search console. Is there really a solution, or Shopify just stopped caring?

ChummyTees
Tourist

Thanks for fixing this. After checking the original thread I started about all this and doing some google searches, this seems to be fixed for all sites. 

 

Here's the original thread:
https://community.shopify.com/c/technical-q-a/shopify-bug-web-pixels-manager-sandbox/m-p/1906981

lilil
Visitor

 

You write

  • Merchant Google Search Console noindex reports should clear after a few months, (we are unable to offer a specific timeframe) as no new pixel URLs should be added. 

 

 

problem is still running. tons of sandbox / wpm url created / last one on nov 25.

Brehmale
Tourist

Ich habe ebenfalls noch 4000 von diesen wpm und webmaster-pixel 404 Seiten in der Search Console - es begann vor 10 Monaten. (Mein Shop hat gerade mal 250 Produkte und ein paar Sammlungen und Seiten). Shopify-Support vor einer Woche: Geduld! Mehr können sie nicht tun. Ich finde es ein Unding. Auch die vorübergehende Entfernung in der Search Console funktioniert nicht. Gibt es noch weitere Lösungsansätze als Warten? Auch wenn das niemand glauben mag, aber zeitgleich zum Entstehen der 404 Seiten ist der Umsatz damals eingebrochen und hat sich bisher nicht mehr erholt. Die 404 Seiten werden immer noch mehr, der Umsatz immer noch weniger. Durch die Empfehlungsseiten habe ich jetzt sogar über 40.000 nicht indexierte Seiten. Ich bin erst seit 11 Monaten bei Shopify und ich hatte einen sehr guten Start nach der Migration - Umsatzziel im Januar 2023 erreicht, plötzlich 404 Fehlerseiten und WUMMS!! Umsatz mitlerweile bei 0,0 angekommen - nur keiner glaubte an einen Zusammenhang. Und das nach 10 erfolgreichen Jahren bei Jimdo (kaum ADS, nur SEO). Es ist mehr als ärgerlich. Vielleicht versteife ich mich zu Unrecht auf die 404 Seiten, dennoch bereue ich es mega, zu Shopify gewechselt zu haben. Man versucht alles in SEO (wenn auch Laienhaft) und dann kommen solche Fehler.

JonathanFC
Shopify Partner

Hi, we are also getting new WPM pages being reported through Google Search Console.

 

Increase in No Index Pages copy.jpg

vanderlvov
Visitor

Same here. Lots of wpm urls piling up since a long time ago ... and still doing it recently. It was not funny to see "the main" thread closed in July while actually the issue has not been resolved.

Authors