I need to edit robots.txt file because, the "https://www.jeremy-hoye.co.uk/collections/unique-engagement-rings" page not getting link juice value. How to modify robots file?
Solved! Go to the solution
This is an accepted solution.
As of today, June 21st, 2021, we have launched the ability to edit the robot.txt file to give merchants more control over the information that is crawled by search engines. You can learn more about how to edit your robot.txt file through our community post here.
Due to the age of the topic, I will be locking this thread and marking it as resolved to help direct anyone that lands on this page. If you have any questions about the new feature, please do not hesitate to create a new post under our "Techincal QA" board.
You can't edit the robots.txt file.
I'm curious though - what would you change to add extra value?
Well it seems Shopify deliberatly undermining clients' effort to have trheir sites proplry indexed @ Goolge, and stubornly refuse to provide a solution.
Why would Shopify ban indexing of i.e. /collections & therefore products , /blogs ? or images ?
This is returned from Google crawler (15 pages indexed, 198 exluded from indexing) :
Warning: 15 Indexed, though blocked by robots.txt
Warning: Google chose different canonical than user
# we use Shopify as our ecommerce platform User-agent: * Disallow: /admin Disallow: /cart Disallow: /orders Disallow: /checkout Disallow: /XXXXXXXXX/checkouts Disallow: /carts Disallow: /account Disallow: /collections/*+* Disallow: /collections/*%2B* Disallow: /collections/*%2b* Disallow: /blogs/*+* Disallow: /blogs/*%2B* Disallow: /blogs/*%2b* Disallow: /*design_theme_id* Disallow: /*preview_theme_id* Disallow: /*preview_script_id* Disallow: /discount/* Disallow: /apple-app-site-association Sitemap: https://XXXXXXX.com/sitemap.xml # Google adsbot ignores robots.txt unless specifically named! User-agent: adsbot-google Disallow: /checkout Disallow: /carts Disallow: /orders Disallow: /XXXXXXXXX/checkouts Disallow: /discount/* Disallow: /*design_theme_id* Disallow: /*preview_theme_id* Disallow: /*preview_script_id* User-agent: Nutch Disallow: / User-agent: MJ12bot Crawl-Delay: 10 User-agent: Pinterest Crawl-delay: 1
Hi there @Dresscheap!
Don here from Shopify. :)
You can learn how to find and submit your sitemap here on our help site.
It would not be necessary to edit your robots.txt file in order to do so.
All the best!
Hi there @jacktyler3!
Don from Shopify once again. :)
If you'd like to add pages to your robots.txt file so that they are not crawled you can do so by following the steps outlined here on our help site.
Note that it is not possible to remove pages from this file, only to add them.
All the best!