Indexation: URL crawling is disallowed in Robots.txt

New Member
1 0 0

I keep having this pop up as an SEO critical error on my blogs. How do I allow robots.txt crawling on selected pages?

0 Likes
Shopify Partner
723 63 97

Would you share the URL to your store as well as the exact error you are getting that would list the URLs affected? Maybe there is discrepancy between sitemap.xml and the content of robots.txt.

Sergiu Svinarciuc | CTO @ visely.io
- Was my reply helpful? Click Like to let me know!
- Was your question answered? Mark it as an Accepted Solution!
- To learn more about the awesome stuff we do head over to visely.io or our blog
0 Likes
Excursionist
13 0 3

it may be because of the blogs are disallowed to crawl and index. So you can fix it by removing those URLs from the robots. if possible share the domain name so that we can have a look at what exact issue is there.

 

0 Likes
Shopify Partner
723 63 97

Small note - there is no way to edit robots.txt for a Shopify store.

Sergiu Svinarciuc | CTO @ visely.io
- Was my reply helpful? Click Like to let me know!
- Was your question answered? Mark it as an Accepted Solution!
- To learn more about the awesome stuff we do head over to visely.io or our blog
0 Likes