I've checked robots.txt syntax and don't see anything.
This is very distressing since it seems to be a recurring issue with Shopify robots.txt files and the site owner is not allowed to make changes to the robots.txt file. Even more so as the home page and the product pages are among those being blocked. Please fix. URL is https://bestketonetest.com/
I am having the same issue here: https://wanuna.store/robots.txt
This is the URL that is blocked and has been excluded (a blog post): https://wanuna.store/blogs/journal/2021-trending-now-fair-trade-home-decor-products
Any suggestion here?
Just a quick update about the robot.txt file that was referenced in this post. As of today, June 21st, 2021, we have launched the ability to edit the robot.txt file to give merchants more control over the information that is crawled by search engines. You can learn more about how to edit your robot.txt file through our community post here.
If you have any questions about the new feature, please do not hesitate to create a new post under our "Techincal QA" board.
Have you created a collection on your online store and experienced an issue with adding yo...By Ollie Aug 24, 2022