What's your biggest current challenge? Have your say in Community Polls along the right column.

/recommendations/products in Google Search Console

Solved

/recommendations/products in Google Search Console

catalyst13
Shopify Partner
1 0 0

I'm getting a warning for the site http://wanderingyogi.com.au/ that /recommendations/products is:

 

Blocked due to other 4xx issue
These pages aren't indexed or served on Google
 
I can't see any links to that URL, it's only appearing in the recommendation block as the data-url that I believe is used to call the API? Couple of questions.
 
1. How is google seeing this URL?
2. Is adding it to robots.txt as blocked the best way to handle it?
3. Any other suggestions to remove the issue?
Accepted Solution (1)

PaulNewton
Shopify Partner
7481 661 1570

This is an accepted solution.

Robots.txt is the way to go

You should first figure out on what pages google is finding that path and figure out what is the source.

Check any sitemaps as well not unlikely someone may have added it to a sitemap if you've had any custom SEO done.

 

Generally that url being crawled should not be happening as the route in most themes is only present javascript and /recommendations/products shouldn't ever be a direct link to that endpoint it would be like cart/clear showing up in results.

 

Unless now goggle bot et al. are checking html tag data attributes for a "url" keyword to infer link, or crawling ajax request endpoints which would be wild but not surprising. 

 

 

 

 

 

Contact paull.newton+shopifyforum@gmail.com for the solutions you need


Save time & money ,Ask Questions The Smart Way


Problem Solved? ✔Accept and Like solutions to help future merchants

Answers powered by coffee Thank Paul with a Coffee for more answers or donate to eff.org


View solution in original post

Replies 2 (2)

PaulNewton
Shopify Partner
7481 661 1570

This is an accepted solution.

Robots.txt is the way to go

You should first figure out on what pages google is finding that path and figure out what is the source.

Check any sitemaps as well not unlikely someone may have added it to a sitemap if you've had any custom SEO done.

 

Generally that url being crawled should not be happening as the route in most themes is only present javascript and /recommendations/products shouldn't ever be a direct link to that endpoint it would be like cart/clear showing up in results.

 

Unless now goggle bot et al. are checking html tag data attributes for a "url" keyword to infer link, or crawling ajax request endpoints which would be wild but not surprising. 

 

 

 

 

 

Contact paull.newton+shopifyforum@gmail.com for the solutions you need


Save time & money ,Ask Questions The Smart Way


Problem Solved? ✔Accept and Like solutions to help future merchants

Answers powered by coffee Thank Paul with a Coffee for more answers or donate to eff.org


Emm_
Shopify Partner
8 0 9

I also have this issue. Google Search Console indicates that https://volupto.com.au/recommendations/products is "Blocked due to other 4xx issue".

One of the referring pages is listed as https://volupto.com.au/products/renoir-stone-bath.

 

Nothing in the page links to that URL, but I found this in the code:

Emm__1-1712896724876.png

 

@PaulNewton would you still recommend adding it to Robots.txt? Or setting up a 301?

 

Thanks in advance for your help!

Emmanuelle.