How can I prevent the new indexing bug from creating useless pages on Google?

Technical Q&A
73

It’s also unclear to me, why Google still indexes those pages. Usually Google does this, when there are Backlinks pointing to that page, but this is not the case. Right now, there is just internal link juice that’s being passed.

With that given, robots.txt and noindex are existing on the same time, which doesn’t work properly, as the noindex is not read because of the robots.txt block.

The way to fix it for most of the people is to make Google read noindex and delete the robots.txt.

  1. delete the "Disallow: /wpm@" from robots.txt
  2. Temporarily remove the URLs from the Google Search console by deleting URL path: /wpm@ This won’t really delete the pages from the index. This can be checked if you go to Google and type in: site:yourdomain.com - the amount of results are still including the indexed web-pixels-pages and will decrease when they are being deleted. Anyway, they’ll be hidden in Google Index, so I’d definitely do that.
  3. Go into GSC and go to indexed pages, even though blocked by robots.txt and start a review. This will just work if in robots.txt, the statement “Disallow: /wpm@” is deleted. You’ll probably need several runs in order to get them all out of the index - depending on how many pages are affected.

Two things still unclear:

  1. Whether noindex/nofollow is enough to prevent the crawling of a large number of URLs. This might be problematic regarding the Crawl Budget, because a lot of pages are crawled, which is not neccesary. Usually, noindexed pages aren’t being crawled as often as indexed pages, but they’re still crawled. Anyway, removing the statement from the robots.txt so that Google can read the noindex is the best way in short term to avoid that those pages are being indexed.
  2. If this might affect SEO due to link juice. It might, but doesn’t neccecarily need to. Possible solutions would need to be procceded by Shopify. Best would be if Shopify would work with Experts in SEO to get the optimal solutions for their vendors - the better the SEO of the Shop-Owners is, the more sales/revenue will be made by Shopify.