Search engines, such as Google, constantly crawl the internet in search of new data as a source for their search results. The robots.txt file tells search engine bots, known as crawlers, which pages to request to view from your online store.
All Shopify stores have a default robots.txt file that's optimized for Search Engine Optimization (SEO). Previously editing this file wasn’t possible but now that’s changed!
With a recent update, it’s now possible to customize your robots.txt from the optimized version that is being used currently in your shop. A few of the edits you might consider include:
If you would like assistance customizing your robots.txt file you will need to hire a Shopify Expert as Shopify Support can not help with the edits you would like to make, nor provide feedback before implementing changes.
Check out the links below to learn more about this new feature, as well as reviewing some examples found in our developer documentation. If you have any questions about this feature leave a reply below!
If you are looking for community feedback on some edits you are looking into, consider posting a detailed message on our Technical Q&A discussion board.